CONVR 2009 Proceedings

CONVR 2009
Proceedings of the 9
th
International Conference on
Construction Applications of Virtual Reality
Sydney, Australia, 5-6 November 2009

Edited by

Xiangyu Wang
The University of Sydney, Australia
Ning Gu
The University of Newcastle, Australia

Sponsors

The University of Sydney

The CONVR 2009 Conference Organising Committee
Xiangyu Wang, The University of Sydney, Australia
Ning Gu, The University of Newcastle, Australia
Michael Rosenman, The University of Sydney, Australia
Anthony Williams, The University of Newcastle, Australia
Nashwan Dawood, The University of Teesside, United Kingdom
ISBN 978-1-74210-145-3
All rights reserved
2009
No part of this work may be reproduced, stored in a retrieval system, or transmitted in any
form or by any means, electronic, mechanical, photocopying, microfilming, recording or
otherwise, without written permission from the Publisher, with the exception of any material
supplied specifically for the purpose of being entered and executed on a computer system,
for exclusive use by the purchaser of the work.

Published and printed at The University of Sydney, Australia,
by University Publishing Service.
The University of Newcastle
The University of Teesside
FORUM8 CO.
TABLE OF CONTENTS

PREFACE vii

CONVR 2009 INTERNATIONAL SCIENTIFIC COMMITEE viii

ACKNOWLEDGEMENTS ix

KEYNOTE SPEECH ABSTRACTS AND SPEAKER BIOS

Construction Synthetic Environments--------------------------------------------------------1
Simaan M. AbouRizk

Seeking How Visualization Makes Us Smart------------------------------------------------3
Phillip S. Dunston

intuBE Project-------------------------------------------------------------------------------------4
Nashwan Dawood

The Role of VR in Improving Collective Intelligence for AEC Processes-------------6
Mary Lou Maher

I. DESIGN COLLABORATION

Virtual Worlds and Tangible Interfaces: Collaborative Technologies
that Change the Way Designers Think-----------------------------------------------------9
Mary Lou Maher, Ning Gu and Mijeong Kim

A Novel Camera-based System for Collaborative Interaction with
Multi-dimensional Data Models--------------------------------------------------------------19
Michael Van den Bergh, Jan Halatsch, Antje Kunze, Frdric Bosch,
Luc Van Gool and Gerhard Schmitt

Towards a Collaborative Environment for Simulation Based Design----------------29
Michele Fumarola, Stephan Lukosch, Mamadou Seck and Cornelis Versteegt

Empirical Study for Testing Effects of VR 3D Sketching on
Designers Cognitive Activities----------------------------------------------------------------39
Farzad Pour Rahimian and Rahinah Ibrahim

Analysis of Display Luminance for Outdoor and Multi-user Use ---------------------49
Tomohiro Fukuda

A Proposed Approach to Analyzing the Adoption and Implementation of
Virtual Reality Technologies for Modular Construction--------------------------------59
Yasir Kadhim, Jeff Rankin, Joseph Neelamkavil and Irina Kondratova

Collaborative 4D Review through the Use of Interactive Workspaces---------------71
Robert Leicht and John Messner

Design Scenarios: Methodology for Requirements Driven
Parametric Modelling of High-rises---------------------------------------------------------79
Victor Gane and John Haymaker

An Experimental System for Natural Collocated and Remote Collaboration--------91
Jian Li and Jingyu Chen

Urban Wiki and VR Applications--------------------------------------------------------------97
Wael Abdelhameed and Yoshihiro Kobayashi

II. AUTOMATION AND INTERACTION

Toward Affective Handsfree Human-machine Interface Approach in
Virtual Environments-based Equipment Operation Training--------------------------107
Iman Mohammad Rezazadeh, Xiangyu Wang, Rui Wang and Mohammad Firoozabadi

Construction Dashboard: An Exploratory Information Visualization Tool
for Multi-system Construction----------------------------------------------------------------117
Cheng-Han Kuo, Meng-Han Tsai, Shih-Chung Kang and Shang-Hsien Hsieh

Computer Gaming Technology and Porosity-----------------------------------------------127
Russell Lowe and Richard Goodwin

Virtual Reality User Interfaces for the Effective Exploration and
Presentation of Archaeological Sites----------------------------------------------------------139
Daniel Keymer, Burkhard Wnsche and Robert Amor

Interactive Construction Documentation----------------------------------------------------149
Antony Pelosi

Case Studies on the Generation of Virtual Environments of
Real World Facilities-----------------------------------------------------------------------------155
Michele Fumarola and Ronald Poelman

Evaluation of 3D City Models Using Automatic Placed Urban Agents----------------165
Gideon Aschwanden, Simon Haegler, Jan Halatsch, Rafal Jeker, Gerhard Schmitt
and Luc van Gool

Integration of As-built and As-designed Models for
3D Positioning Control and 4D Visualization during Construction---------------------177
Xiong Liang, Ming Lu and Jian-Ping Zhang

Augmenting Site Photos with 3D As-built Tunnel Models for
Construction Progress Visualization----------------------------------------------------------187
Ming Fung Siu and Ming Lu

Automatic Generation of Time Location Plan in Road Construction Projects------197
Raj Kapur and Nashwan Dawood

Development of 3D-Simulation Based Genetic Algorithms to
Solve Combinatorial Crew Allocation Problems-------------------------------------------207
Ammar Al-Bazi, Nashwan Dawood and John Dean

Integration of Urban Development and 5D Planning--------------------------------------217
Nashwan Dawood, Claudio Benghi, Thea Lorentzen and Yoann Pencreach

III. SIMULATION AND ANALYSIS

A Simulation System for Building Fire Development and
the Structural Response due to Fire----------------------------------------------------------229
Zhen Xu, Fangqin Tang and Aizhu Ren

Physics-based Crane Model for the Simulation of Cooperative Erections-----------237
Wei Han Hung and Shih Chung Kang

Interaction between Spatial and Structural Building Design:
A Finite Element Based Program for the Analysis of
Kinematically Indeterminable Structural Topologies-------------------------------------247
Herm Hofmeyer and Peter Russell

Virtual Environment on the Apple iPhone/iPod Touch-----------------------------------257
Jason Breland and Mohd Fairuz Shiratuddin

3D Visibility Analysis in Virtual Worlds: The Case of Supervisor---------------------267
Arthur van Bilsen and Ronald Poelman

Evaluation of Invisible Height for Landscape Preservation
Using Augmented Reality-----------------------------------------------------------------------279
Nobuyoshi Yabuki, Kyoko Miyashita and Tomohiro Fukuda

An Experiment on Drivers Adaptability to Other-hand Traffic
Using a Driving Simulator----------------------------------------------------------------------287
Koji Makanae and Maki Ujiie

C2B: Augmented Reality on the Construction Site----------------------------------------295
Lon van Berlo, Kristian Helmholt and Wytze Hoekstra

Development of a Road Traffic Noise Estimation System
Using Virtual Reality Technology-------------------------------------------------------------305
Shinji Tajika, Kazuo Kashiyama and Masayuki Shimura

Application of VR Technique to Pre- and Post-Processing for
Wind Flow Simulation in Urban Area--------------------------------------------------------315
Kazuo Kashiyama, Tomosato Takada, Tasuku Yamazaki, Akira Kageyama,
Nobuaki Ohno and Hideo Miyachi

Construction Process Simulation Based on Significant Day-to-day Data-------------323
Hans-Joachim Bargstdt and Karin Ailland

Effectiveness of Simulation-based Operator Training------------------------------------333
John Hildreth and Michael Stec

IV. BUILDING INFORMATION MODELLING

BIM Server: Features and Technical Requirements--------------------------------------345
Vishal Singh and Ning Gu

LEED Certification Review in a Virtual Environment-----------------------------------355
Shawn OKeeffe, Mohd Fairuz Shiratuddin and Desmond Fletcher

Changing Collaboration in Complex Building Projects

through the Use of BIM-------------------------------------------------------------------------363
Saskia Gabril

The Introduction of Building Information Modelling in Construction Projects:
An IT Innovation Perspective-----------------------------------------------------------------371
Arjen Adriaanse, Geert Dewulf and Hans Voordijk

Creation of a Building Information Modelling Course for
Commercial Construction at Purdue University------------------------------------------383
Shanna Schmelter and Clark Cory

vii
Preface

The Faculty of Architecture, Design and Planning, at the University of Sydney and the
School of Architecture and Built Environment, at the University of Newcastle are proud to
co-host CONVR 2009, the 9
th
International Conferences on Construction Applications of
Virtual Reality. Significantly, the conference is the 9
th
gathering of this international body of
scholars and professionals across all Architecture, Engineering and Construction (AEC)
disciplines, who dedicate and contribute to the knowledge building and applications of a
broad range of advanced visualisation technologies in the AEC industry. Although the name
of the conference has Virtual Reality (VR) as the keyword, it actually covers much broader
range of visualisationrelated topics beyond VR, which makes the range of the audiences
wider with broader impact as years go.

The CONVR 2009 conference has attracted much attention and recognition among the
research and professional communities involved in the AEC industry. The organising
committee received close to 70 abstracts. After two rounds of rigorous double blind reviews
(the first round for abstract review and the second round for full paper review) by
International Scientific Committee, the CONVR 2009 is very pleased to accept 39 high
quality full papers in this volume.

The CONVR 2009 conference provides a unique platform for experts in the fields to report,
discuss and exchange new knowledge, which has resulted from the most current research
and practice of advanced visualisation technologies. The Organising Committee is pleased
to present selected papers that highlight the state-of-the-art development and research
directions across the following four themes:
Design Collaboration
Automation and Interaction
Simulation and Analysis
Building Information Modelling

In the following pages, you will be able to find a range of quality papers that truly capture
the quintessence of these concepts and will certainly challenge and inspire readers.

The CONVR 2009 Conference Organising Committee:
Xiangyu Wang (Chair), The University of Sydney, Australia
Ning Gu (Co-chair), The University of Newcastle, Australia
Michael Rosenman (Co-chair), The University of Sydney, Australia
Anthony Williams, The University of Newcastle, Australia
Nashwan Dawood, The University of Teesside, United Kingdom

November 2009
viii
CONVR 2009 International Scientific Committee

Karin Ailland Bauhaus-University Weimar
Robert Amor The University of Auckland
Serafim Castro The University of Teesside
Chiu-Shui Chan Iowa State University
Clark Cory Purdue University
Robert Cox Purdue University
Nashwan Dawood The University of Teesside
Paulo Dias Instituto de Engenharia Electrnica e Telemtica de Aveiro (IEETA)
Ning Gu The University of Newcastle, Australia
Ftima Farinha EST- Algarve University Portugal
Michele Fumarola Delft University of Technology
Jan Halatsch ETH Zurich
David Heesom The University of Wolverhampton
Wei-Han Hung National Taiwan University
Rahinah Ibrahim University Putra Maylaysia
Vineet Kamat The University of Michigan
Jeff Kan Taylor College Malaysia
Shih-Chung Kang National Taiwan University
MiJeong Kim Kyung Hee University
Robert Lipman National Institute of Standards and Technology (NIST)
Russell Lowe The University of New South Wales
Koji Makanae Miyagi University
John Messner Penn State University
Esther Obonyo The University of Florida
Svetlana Olbina The University of Florida
Aizhu Ren Tsinghua University
Enio Emanuel Ramos Russo Catholic University in Rio
Iman Rezazadeh Islamic Azad University
Michael Rosenman The University of Sydney
Marc Aurel Schnabel Hong Kong Chinese University
Mohd Fairuz Shiratuddin The University of Southern Mississippi
Augusto de Sousa Universidade do Porto
Andrew Strelzoff Brown University
Walid Tizani The University of Nottingham
Xiangyu Wang The University of Sydney
Vaughn Whisker Penn State University
Antony Williams The University of Newcastle, Australia
ix
ACKNOWLEDGEMENTS
We express our gratitude to all authors for their enthusiasms to contribute their research as
published in this proceedings. Furthermore, this proceedings would not have been possible
without the constructive comments and advice from all the International Scientific
Committee members. We are also deeply grateful to the other members on the organising
committee, Dr. Michael Rosenman, Professor Anthony Williams, and Professor Nashwan
Dawood. Thanks and appreciation specifically goes to Ms Rui Wang for designing our
proceedings and CD covers. We are also grateful to the conference assistants Ms Mercedes
Paulini, Mr Lei Hou and Mr Wei Wang, whose great backup support is essential for the
success of the conference. Financial aid came from Design Lab at the Faculty of
Architecture, Design and Planning at the University of Sydney, School of Architecture and
Built Environment at the University of Newcastle Australia, and Forum8 Co.
KEYNOTE SPEECH 1
Dr. Simaan M. AbouRizk, Professor and NSERC Industrial Research Chair in Construction Engineering
and Management Canada Research Chair in Operation Simulation Department of Civil and Environmental
Engineering, University of Alberta, Canada.
Construction Synthetic Environments
The presentation describes our vision for a highly
integrated, interoperable, distributed simulation
framework for modeling and analyzing construction
projects.
We first describe the evolution of simulation
applications (over a period of 15 years) within the
construction industry in Alberta by providing the
attendees with an overview of select implementation
of simulation-based systems in industrial
applications. Systems were introduced through
collaborations between major construction companies
and the University of Alberta. Those systems were
deployed by the partner companies in different ways,
including planning for tunnel construction projects,
scheduling of modules in a module yard with space
constraints for an industrial contractor, analysis of
fabrication shops for improvement, process
improvement studies, and others.
The presentation then provides an overview of our
vision of advanced simulation systems we call
Construction Synthetic Environments (COSYE), the
intent of which will be to achieve a fully integrated, highly automated construction execution environment
across all project phases and throughout the facilitys life cycle, as articulated in Figure 1. The figure
demonstrates a large-scale distributed simulation framework that provides a comprehensive representation of
an entire construction project with all of its components, including: a model of the facility (product model), the
production/construction operations (process models), the business models, the resources involved, and the
environment under which the project takes place. The framework allows the simulation models to extend
throughout the life of the project with real-time input and feedback to manage the project until it is handed over
to operations. The goal is to provide a virtual world where a construction project is planned, executed, and
controlled with minimum disruption to the actual project. The framework will provide means to establish:
detailed and comprehensive modeling of the entire life cycle of facilities;
collaboration amongst a variety of stakeholders in building the required virtual
models that represent the project; seamless integration between various forms of
simulation (discrete, continuous, heuristic, etc.) and simulation software and tools;
reusable simulation components for many applications (e.g. weather generation,
equipment breakdown processes etc); and man-machine interactions with the
models.
We have completed three prototype synthetic environments using the COSYE framework over the past few
years, including ones for industrial construction, steel construction, a bidding game, and tunnel construction.
We will provide an overview of these during the presentation and select environment to demonstrate in greater
detail.
TheConstructionSyntheticEnvironmentFramework
Bio of Dr. AbouRizk:

Dr. AbouRizk currently holds the positions of Canada Research Chair in Operation Simulation and the
Industrial Research Chair in Construction Engineering and Management in the Department of Civil and
Environmental Engineering at the University of Alberta. He received his PhD degree from Purdue University
in 1990 and his MSCE from Georgia Tech in 1985. He joined the University of Alberta in 1990 and was
promoted to full professor in July 1997.
Dr. AbouRizks research accomplishments have been recognized through numerous awards for the quality of
his research in the field of construction engineering and management, including the prestigious ASCE Peurifoy
Construction Research Award, the E.W.R. Steacie Memorial Fellowship from the Natural Sciences and
Engineering Research Council of Canada, the Thomas Fitch Rowland Prize for best paper in construction
engineering management, the Killam Professorship, the Walter Shanly Award, and the E. Whitman Wright
Award.
Dr. AbouRizk has led the development of the Hole School of Construction Engineering at the University of
Alberta into one of the most reputable construction engineering and management programs in North America,
boasting global recognition for the success of its graduate students and the strength of its faculty members. The
success and distinctiveness of this program are based on strong industry collaboration in the areas of research,
teaching, and overall practice. Dr. AbouRizks method has garnered wide support from funding agencies,
policy makers, and industry practitioners, and has attracted some of the brightest students from around the
world. He is renowned in the academic construction community for his research in computer simulation and its
applications in construction planning, productivity improvement, constructability reviews and risk analysis.
2
KEYNOTE SPEECH 2

Dr. Phillip S. Dunston, Associate Professor in the Division of Construction Engineering and Management,
School of Civil Engineering, at Purdue University, USA.
Seeking How Visualization Makes Us Smart
It was in 1996 that a computer science researcher
brought a vision for applying Augmented Reality
visualization technology to the attention of
attendees at a civil engineering computing
conference. The door was then opened for a new
set of inquiring minds to join architects who were
already taking a look at the possibilities of virtual
visualization. The exciting visualization
opportunities presented by Virtual Reality and
Mixed Reality technologies have since captured the
attention of a growing number of researchers from
the broad architecture, engineering, construction
and facilities management (AEC/FM) domain.
Unlike computer science and computer engineering
researchers who have a technology development
perspective, the AEC/FM community has a user
perspective that must be developed as part of our
contribution to shaping the development of these
technologies and ultimately realizing their adoption
into practice. Opportunities exist for improving
practice through new efficiencies and through
devising new ways of executing work tasks. Our
attention to human resource capabilities and the
attendant human factors can yield successful technology development decisions and integration. This keynote
talk will review our experience in exploring this softer side as well as how we have inevitably had to confront
the more technical challenges and will also suggest how some future objectives might be pursued.

Bio of Dr. Dunston:
Phillip S. Dunston, an Associate Professor with appointments in the Division of Construction Engineering and
Management and the School of Civil Engineering at Purdue University in West Lafayette, Indiana, USA. He is
a 2003 US National Science Foundation Career grantee for research on Mixed Reality applications for the
architecture, engineering and construction (AEC) industry. He directs the Advanced Construction Systems
Laboratory (ACSyL) and is a Co-Director of the Center for Virtual Design of Healthcare Environments, both at
Purdue. His research emphasizes the human factors related to virtual visualization and applying such principles
in specifying the features and functions of visualization systems.
3
KEYNOTE SPEECH 3
Prof. Nashwan Dawood, Director for the Centre for Construction Innovation & Research,
University of Teesside and Cecil M Yuill Professor of Construction management & IT, UK.
intUBE Project Intelligent Use of Buildings
Energy Information (www.intube.eu)
It is a well established fact that buildings are one of the
major contributors to energy use and CO2 emissions.
The energy used in buildings accounts for 40 % of the
total energy use in Europe. While some breakthroughs
are expected in new buildings, the pace of these
improvements is too slow considering the EU's
ambitious goal to improve energy efficiency by 20 %
before 2020. With over 80% of the European buildings
standing in 2020 being already built, the main aim of
the IntUBE project is to develop and make use of
information and communications technologies
including Virtual Reality to improve the energy
efficiency of these existing buildings in compliance
with the EU's aims of improving energy efficiency.
IntUBE will develop tools for measuring and analysing
building energy profiles based on user comfort needs.
These will offer efficient solutions for better use and
management of energy use within buildings over their
lifecycles. Intelligent Building Management Systems
will be developed to enable real-time monitoring of
energy use and optimisation. They will, through
interactive visualisation of energy use, offer solutions
for user comfort maximisation and energy use
optimisation.
intUBEconcept
Bio of Prof. Dawood:

Prof. Dawood is the director for Construction Innovation & Research at the unversity of Teesside and hold
Yuil Professor Chair. Prof Dawood has spent many years as an academic and researcher within the field of
construction management and the application of IT in the construction process. This has ranged across a
number of research topics including information technologies and systems (4D,VR,Integrated databases), risk
management, and business processes. This has resulted in over 170 published papers in refereed international
journal and conferences, and research grants from British Council, Industry, Engineering Academy , EPSRC,
DTI and construction industry companies, totalling about 2,500,000. Final reports of the last three EPSRC
grants received Tending to Outstanding' peer assessment review from EPSRC.
I have been a visiting fellow/Professor at VTT -Finland, University of Calgary- Canada, University of Bahrain-
Bahrain, Central University of Taiwan, AIT- Thailand, Stanford University-USA, PWRI (Public Works
Research Institutes)- Japan, Georgia Tech- USA, Virginia Tech- USA, UNSW- Australia, University of
Parana- Brazil, University of Florida-USA, International Islamic University Malaysia, Gyeongsang National
University, Korea and Miyagi university , Japan and Osaka University, Japan.
Prof. Dawood has originated the CONVR conference series (Construction Applications of Virtual Reality:
Current Initiatives and Future Challenges). The mission of this is to bring together national and international
researchers and practitioners from all areas of the construction industry and promote efficient exchange of
ideas and develop mutual understanding of needs and potential applications of VR modelling. CONVR 2000
was organised at Teesside and attended by participants from 9 countries, CONVR 2001 organised at Chalmers
University, Sweden and attended by participants from 12 countries. CONVR 2003 was organised by Virginia
Tech, USA, CONVR 2004 was organised by ADETTI (Portugal), CONVR 2005 organised in Durham UK,
CONVR 2006 was organised by Florida State University and CONVR 2007 was organised at Penn State
University, USA, CONVR 2008 was organised by IIUM Malaysia and CONVR 2009 will be organised by the
University of Sydney, Australia.

KEYNOTE SPEECH 4

Prof. Mary Lou Maher, the Deputy Division Director of the Information and Intelligent Systems Division
at National Science Foundation and Professor at the University of Sydney, Australia.
The Role of VR in Improving Collective Intelligence for AEC Processes
Collective intelligence is a kind of intelligence that
emerges from the collaboration and competition of
individuals. While the concept of collective intelligence
has been around for a long time, recent renewed interest
in collective intelligence is due to internet technologies
that allow collective intelligence to emerge from
remotely located and potentially very large numbers of
individuals. Wikipedia is a product of collective
intelligence as a source of knowledge that is
continuously generated and updated by very large
numbers of individuals. Similarly, Second Life is a
product of collective intelligence as a 3D virtual world
that is created and modified by the large numbers of
individuals that enter the world. The AEC industry relies
on the collective intelligence of many individuals and
teams of professionals from different disciplines. Virtual
reality in its many forms provides collaborative
technologies that not only enable people to work
together from a distance but also change the way we
interact with each other and the shared digital models
that comprise the product of the collaboration. This
presentation presents a new kind of collective
intelligence enabled by emerging technologies, the
research challenges, and the potential impact on design
and creativity in AEC projects.
Bio of Prof. Maher:
Mary Lou Maher is the Deputy Division Director of the Information and Intelligent Systems Division at NSF.
She joined the Human Centered Computing Cluster in July 2006 and initiated a funding emphasis at NSF on
research in creativity and computing called CreativeIT. She is the Professor of Design Computing at the
University of Sydney. She received her BS (1979) at Columbia University and her MS (1981) and PhD (1984)
at Carnegie Mellon University. She was an Associate Professor at Carnegie Mellon University before joining
the University of Sydney in 1990. She has held joint appointments in the Faculty of Architecture and the
School of Information Technologies at the University of Sydney. Her own research includes empirical studies
and new technologies for design in virtual worlds and other collaborative environments, behavior models for
intelligent rooms, motivated reinforcement learning for non-player characters in MMORPGs, and tangible user
interfaces for 3D design.
6

DESIGN COLLABORATION

Virtual Worlds and Tangible Interfaces: Collaborative Technologies
That Change the Way Designers Think-----------------------------------------------------9
Mary Lou Maher, Ning Gu and Mijeong Kim

A Novel Camera-based System for Collaborative Interaction with
Multi-dimensional Data Models--------------------------------------------------------------19
Michael Van den Bergh, Jan Halatsch, Antje Kunze, Frdric Bosch,
Luc Van Gool and Gerhard Schmitt

Towards a Collaborative Environment for Simulation Based Design----------------29
Michele Fumarola, Stephan Lukosch, Mamadou Seck and Cornelis Versteegt

Empirical Study for Testing Effects of VR 3D Sketching on
Designers Cognitive Activities----------------------------------------------------------------39
Farzad Pour Rahimian and Rahinah Ibrahim

Analysis of Display Luminance for Outdoor and Multi-user Use ---------------------49
Tomohiro Fukuda

A Proposed Approach to Analyzing the Adoption and Implementation of
Virtual Reality Technologies for Modular Construction--------------------------------59
Yasir Kadhim, Jeff Rankin, Joseph Neelamkavil and Irina Kondratova

Collaborative 4D Review through the Use of Interactive Workspaces---------------71
Robert Leicht and John Messner

Design Scenarios: Methodology for Requirements Driven
Parametric Modelling of High-rises---------------------------------------------------------79
Victor Gane and John Haymaker

An Experimental System for Natural Collocated and Remote Collaboration-------91
Jian Li and Jingyu Chen

Urban Wiki and VR Applications-------------------------------------------------------------97
Wael Abdelhameed and Yoshihiro Kobayashi

9
th
International Conference on Construction Applications of Virtual Reality Nov 5-6, 2009

VIRTUAL WORLDS AND TANGIBLE INTERFACES: COLLABORATIVE
TECHNOLOGIES THAT CHANGE THE WAY DESIGNERS THINK
Mary Lou Maher, Professor,
University of Sydney, Australia;
mary@arch.usyd.edu.au, http://web.arch.usyd.edu.au/~mary
Ning Gu, Lecturer,
University of Newcastle, Australia;
ning.gu@newcastle.edu.au, http://www.newcastle.edu.au/school/arbe
Mi Jeong Kim, Lecturer,
Kyung Hee University, Korea;
mijeongkim@khu.ac.kr, http:// housing.khu.ac.kr
ABSTRACT: Reflecting on the authors computational and cognitive studies of collaborative design, this paper
characterizes recent research and applications of collaborative technologies for building design. The specific
technologies considered are those that allow synchronous collaboration while planning, creating, and editing 3D
models, including virtual worlds, augmented reality (AR) tabletop systems, and tangible user interfaces (TUIs).
Based on the technical capabilities and potential of the technologies described in the first part, the second part of
the paper considers the implications of these technologies on collaborative design based on an overview of the
results of two cognitive studies conducted by the authors. Two studies, using protocol analysis, are described as the
basis for characterizing the designers cognitive actions, communication and interaction in different collaborative
design situations. The first study investigated collaborative design in a virtual world to better understand the
changes in design behavior when the designers are physically remote but virtually collocated as avatars in a 3D
model of their design solution. The second study measured the effects of tangible user interfaces (TUIs) with AR on
a tabletop system on designers cognitive activities and design process in co-located collaboration. The paper
concludes by discussing the implications of the results of these studies on the future design of collaborative
technologies for designers.
KEYWORDS: Collaborative Design, 3D Virtual Worlds, Tangible Interfaces, Protocol Analysis, Design Cognition.
1. COLLABORATIVE TECHNOLOGIES AND DESIGN
Collaborative design is a process of dynamically communicating and working together within and across disciplines
in order to collectively establish design goals, search through design problem spaces, determine design constraints,
and construct a design solution. While each designer contributes to the development of the design solution,
collaboration implies teamwork, negotiation, and shared models. Collaborative technologies support design in
several ways, two of which are (1) the ability for collaboration to occur at the same time while the participants are
remotely located and (2) the ability to augment the perception of the shared design drawings or models through new
technologies for interacting with digital models. In this paper we show how two specific collaborative technologies
change the way designers think.
We focus on the architectural design of buildings where eventually the design solution is a model of a 3D product
that evolves as the record and the focus of the design process. Bringing designers into 3D virtual environments has
the potential to improve their understanding of the design models during the collaborative process. Two such
environments, 3D virtual worlds and tangible user interfaces (TUIs) to 3D models, are very different approaches to
making the design model accessible to remote and collocated designers. Various virtual worlds and tangible
interaction technologies have been developed for the AEC (Architecture, Engineering and Construction) domain, but
most of them are still in the lab-based prototype development and validation stages. In this paper we reflect on the
potential and implications of these collaborative technologies from a cognitive perspective in order to understand
their role in design practice and to contribute a cognitive basis for the design of new collaborative technologies.
9
9
th

2. BACKGROUND
The background section reviews recent developments in collaborative technologies and introduces a research
method - protocol analysis - for studying design cognition in collaborative design.
2.1 Developments in collaborative technologies for design
Collaborative technologies for design allow two or more designers to create shared drawings or model and design
together while remote or collocated. Since the focus while designing is on the shared drawings or models of the
design, the important aspects of collaborative technologies for design are: the type of digital media available to
represent the design, the interaction technologies for creating, visualizing, and modifying the shared drawings or
models, and the ways in which the designers communicate and interact with each other.
Research into digital technologies for supporting collaborative work started in the 1960s with the early work at
Stanford Research Institute into innovative interaction techniques. Later the developments at Xerox PARC in the
1970s and 1980s brought the field to what became known as user-centered design (Norman and Draper 1986) an
important research focus in an emerging field called Human Computer Interaction (HCI). In the early 1990s, a
branch of HCI developed into the research area of Computer Supported Cooperative Work (CSCW) and groupware
technologies. Like HCI, CSCW is a multi-disciplinary field that has made significant contributions to the design of
collaborative technologies. All these technological developments changed the practice of many fields such as
science, art and design. In terms of functions, the developments in computer-supported technologies for
collaborative design can be classified into the categories of video conferencing, shared drawings, and shared models.
The types of digital media for design representation can include bit mapped images, sketches, structured graph
models, 2D and 3D geometric CAD models, and 3D object oriented models. Some commonly used technologies for
supporting the collaborative design of buildings are digital sketching tools such as GroupBoard
(http://www.groupboard.com), Sketchup (http://sketchup.google.com), and the major CAD systems. The interaction
technologies include the standard graphical user interfaces (GUI) using keyboard and mouse, touch screens and
tables such as the Microsoft Surface (http://www.microsoft.com/surface/), augmented reality, and game controllers
such as the Nintendo Wii (http://wii.com). Designers can communicate with each other using text chat, voice over
IP, and/or video. With the technical advances in the field and the wider adoption of high-bandwidth internet, new
generation of collaborative technologies have emerged, two of which including 3D virtual worlds and TUIs to 3D
models are the focus of our paper.
3D virtual worlds are networked environments designed using the place metaphor. One of the main characteristics
that distinguish 3D virtual worlds from conventional virtual reality is that 3D virtual worlds allow multiple users to
be immersed in the same environment supporting a shared sense of place and presence (Singhal and Zyda 1999).
Multi-user 3D virtual worlds have grown very rapidly, with examples such as Second Life
(http://www.secondlife.com) having reached millions of residents and boasting a booming online economy. Through
the use of the place metaphor, 3D virtual worlds have been associated with the physical world ever since the early
conceptual formation of the field. On one hand, the rich knowledge and design examples in the physical world
provide a good starting point for the development of 3D virtual worlds. On the other hand, designing in 3D virtual
worlds is becoming an exciting territory for the new generation of designers to explore. For the AEC industry, recent
developments in 3D virtual worlds and the proliferation of high bandwidth networked technologies have shown
great potential in transforming the nature of remote design collaboration. In 3D virtual worlds, designers can
remotely collaborate on projects without the barriers of location and time differences. With high-speed network
access, real-time information sharing and modifications of large data sets such as digital building models become
possible over the World Wide Web. Distant design collaboration can significantly reduce the relocation costs and
help to increase efficiency in global design firms. Current development of such systems, for example, DesignWorld
(Maher et al. 2006a) supports remote communication, collaborative 3D modeling and multidisciplinary building
information sharing.
TUIs couple physical artifacts and architectural surfaces to the correlated digital information, whereby the physical
representations of digital data serve simultaneously as interactive controls. For design applications, TUIs are often
combined with AR, which allows designers to explore design alternatives using modifiable models. Through the
direct tangible interaction, designers can use their hands and often their entire bodies for the physical manipulation.
More recently, a wide range of tangible input devices have been developed for collaborative design (Deitz and Leigh
10
9
th

2001; Fjeld et al. 1998; Ishii and Ullmer 1997; Moeslund et al. 2002). BUILD-IT provides physical 3D bricks for
concurrently accessing to multiple digital models and the ARTHUR system employs a wand for interacting with
3D virtual settings projected into their common working environment. The metaDESK embodies many of the
metaphorical devices of windows, icons, and menus of GUIs as a physical instantiation such as a lens with wooden
frames, phicons, and trays respectively. The InteracTable and DiamondTouch use touch-sensitive displays for
supporting cooperative team work. These tangible input devices support multi-user interactions for co-located or
remote collaboration by providing a sensory richness of meaning. Tangible interaction is recently becoming an
important research issue in the domain of design computation. However, compared to the rapid development of
tangible interaction technologies, relatively little is known about the effects of tangible interaction systems on design
cognition.
2.2 Studying design cognition in collaborative design: protocol analysis
While usability is a critical feature of collaborative technologies, usability studies do not necessarily provide a
critical assessment of the impact of technology on design processes. Rather than focus on usability, studying the
perception, actions, and cognition of designers using collaborative technologies allows us to compare the
technologies and their impact on human behavior while designing. Protocol analysis has been used to study how
people solve problems, and has been used widely in the study of design cognition (Gero and Mc Neill 1997; Suwa et
al. 1998). We adapt this method to study perception, action, and cognition while designers are using collaborative
technologies.
A protocol is the recorded behavior of the problem solver which is usually represented in the form of sketches,
notes, video or audio recordings. Whilst the earlier studies dealt mainly with protocols verbal aspects, later studies
acknowledge the importance of design drawing, associating it with design thinking which can be interpreted through
verbal descriptions (Stempfle and Badke-Schaub 2002; Suwa and Tversky 1997). Recent design protocol studies
employ analysis of actions which provide a comprehensive picture of physical actions involved during design
(Brave et al. 1999). In design research, two kinds of protocols are used: concurrent protocols and retrospective
protocols. Generally, concurrent protocols are collected during the task and utilized when focusing on the process-
oriented aspect of designing, being based on the information processing view (Simon 1992). The think-aloud
technique is typically used, in which subjects are requested to verbalize their thoughts as they work on a given task
(Ericsson and Simon 1993; Lloyd et al. 1995). Retrospective protocols are collected after task and utilized when
focusing on the content-oriented aspects of design, being concerned with the notion of reflection in action (Dorst
and Dijkhuis 1995; Schn 1983).
The methodology involves developing an experiment in which one or more designers are asked to work on a design
task while being recorded. The recording, or protocol data, is a continuous stream of data and can include video of
the designers, and/or continuous video of the computer display showing the designers actions and an audio stream
of the verbalization of the designer(s). The protocol data is segmented into units that are then coded and analyzed to
characterize the design session. The coding scheme is developed according to the theory, model, or framework that
is being tested and can include cognitive, communication, gesture, or interactive actions.
The protocol analysis technique has been adopted to understand the interactions of design teams (Cross and Cross
1996; Stempfle and Badke-Schaub 2002) and design behavior of teams (Goldschmidt 1996; Valkenburg and Dorst
1998). Protocol studies of collaborative architectural design focus on understanding team collaboration, in terms of
use of communication channels and design behavior variables (Gabriel and Maher 2002). Protocol coding has been
conducted on professional architects and students architects respectively for the two studies described below in
Sections 3 and 4. For the two studies, the think aloud method is not directly applicable in the protocol collection.
The protocol data comprises the designers conversations, gestures, and interactions rather than the designers
verbalization of their thoughts as in the think aloud method. Such collaborative protocols provide data indicative of
cognitive activities that are being undertaken by the designers, not interfering with design process as a natural part of
the collaborative activities.
3. DESIGNING IN A 3D VIRTUAL WORLD
This study compares the collaborative design process in a 3D virtual world to collaborative design processes in a
traditional face-to-face sketching environment and in a remote sketching environment. We set up three distinctive
11
9
th

design environments and studied four pairs of professional architects
1
each collaborating on a different design task
with similar complexity in each of the three design environments. The comparison of the same pairs of designers in
three different environments is assumed to provide a better indication of the impact of the environments on design
cognition than using different designers or a same design task. A more detailed description this study is reported in
(Maher et al. 2006b).
3.1 Experiment setup and coding scheme design
A different design brief and a collage of the photos showing the site and the surroundings were provided for each of
the three different experiment sessions. In the face to face session of the experiment, the pairs of designers used pen
and paper as the sketching tools. In the remote sketching session, they sketched remotely and synchronously, with
one designer using a Smart Board (http://www.smarttech.com) and the other using Mimio (http://www.mimio.com).
Both technologies provide a pen and digital ink interface. In the final 3D virtual world session, designers
collaborated remotely and synchronously in a 3D virtual world - Active Worlds (http://www.activeworlds.com) -
through 3D design and modeling. In the latter two sessions of the experiment, remote and synchronous
communication was simulated by locating both designers in two different parts of the same room, allowing them to
talk to each other, but only seeing each other via web cams. Each session required the designers to complete the
design task in 30 minutes. They were given training sessions on the use of Smart Board, Mimio and Active Worlds
prior to the experiment.
The basis of the coding scheme design for the research is a consideration of a set of expected results. We developed
and applied a five-category coding scheme including communication content, design process, operations on external
representations, function-structure, and working modes. The communication content category partitions each
session according to the content of the designers conversations, focusing on the differences in the amount of
conversation devoted to discussing design development when compared to other topics. The design process category
characterizes the different kinds of designing tasks that dominate in the three different design environments. The
operations on external representation category look specifically at how the designers interacted with the external
design representation to see if the use of 2D sketches or 3D models was significantly different. The function-
structure category further classifies the design-related content as a reference to the function of the design or the
structure of the design. The working modes category characterizes each segment according to whether or not the
designers were working on a same design task or on a same part of the design representations.
3.2 Protocol analysis result
The analysis of collaborative design behavior involves documenting and comparing the categories of codes. We
looked at frequencies of the occurrence of the code categories in the three different sessions. We also documented
the time spent for each category, with respect to the total time elapsed during the session. This data gives us the
duration percentages of the codes in each main category. Table 1 provides an overview of the focus of activity in
each of the three design environments by showing the average percentages for 4 of the 5 coding categories of the
pairs of the designers who participated in the experiment. We dont show the working mode category in TABLE 1
because it will always be a total of 100% of the duration since the designers are always either working on the same
or different tasks. The categories of codes were applied independently therefore each segment could be coded in
more than one category.
TABLE. 1: Durations of codes in each main category as average percentages of the total elapsed time.
Categories Face to Face Sketching Remote Sketching 3D Virtual Worlds
Communication Content
72% 72% 61%
Design Process
69% 48% 34%
Operations on External Representations
96% 90% 93%
Function-Structure
67% 43% 27%

1
While four pairs of designers are not considered a statistically significant number of participants in a cognitive study, we can
use these results as an exploratory study to identify major differences that are common across this sample of designers.
12
9
th

We tested if there are significant differences between the pairs across the three different design sessions in terms of
the design behavior (coded activity categories). The ANOVA test (ANOVA with replication, P<0.05 between the
three different design sessions) result shows that there is no significant difference between the pairs in terms of
communication content (p=0.15), operations related to external representations (p=0.80) and working mode
(p=0.99). The results are listed in TABLE 2. These results support that the collaborative behavior in these categories
did not show a significant variance amongst the different pairs. Note that the design process and function-structure
categories are significantly different between the pairs. These differences are not surprising and they are common in
design studies since design activities of a particular designer can change due to different situations in the actual
context, and the variance in individual design strategies can have an effect on the collaborative design process.
TABLE. 2: ANOVA test result on action categories of different designer pairs.
P-value
Communication
Content
Design
Process
Operations on External
Representations
Function-
Structure
Working
Mode
Between designer pairs in each design session 0.15 2.46E-08 0.80 3.26E-05 0.99
The details of the protocol analysis, reported in (Maher et al. 2006b), show that there are insignificant differences
among the three design environments, in terms of the communication content and operations on external
representations categories. 3D virtual worlds are able to support design communication and representation during
collaboration. There is a significant decrease from face to face sketching to remote sketching and to 3D design and
modeling in virtual worlds, in terms of the design process and function-structure categories. The first interpretation
of such differences is that during remote collaboration, designers are able to design both collectively and
individually due to the flexibility of digital media in modifying and integrating different parts or different versions
of the design representations, as well as the physical separation of the designers. The evidence for this is the results
of the working mode category, showing a significant difference among the face to face and remote design
environments. It was observed in our experiment that during the face to face sessions, designers mostly worked
together with over 95% of the duration devoted to collaborative mode. Although sometimes a particular designer led
the process while the other observed and critiqued, they always focused on the same tasks. In the remote sessions,
and especially in 3D virtual worlds, an average 40% of the duration was for individual design phases where different
designers worked on different tasks or different parts of the design representations. They often came together after
an individual phase to review each others outcomes or swap tasks. During these individual phases, they reduced and
some pairs even stopped verbal communications (note the decrease in the communication content category for 3D
virtual worlds). This explains the smaller number of design-related segments during the remote sketching and 3D
virtual world sessions. The second interpretation of such differences is the possible changes in the approach to
design development when switching from 2D sketching to 3D modeling. In sketching sessions, although designers
constantly externalize their design ideas in separated sketch parts or over other existing sketches, there is usually a
clear separation between the development of design concepts and the development of formal design representations.
In 3D virtual worlds, these two processes become more blurry. The 3D virtual world objects designers used to
explore concepts also become the 3D models for the final design representations. The design ideas are evolved,
explored and externalized all through 3D modeling. Our current perceptions about 3D modeling and the current
setups of the experiment are inadequate to further understand the different roles of 3D modeling in design
collaboration. Future research is needed in this regard.
A further analysis on each of the coded categories was also conducted and the main points are summarized below:
(1) the communication about awareness of the other designer increases during remote collaboration with the highest
duration percentage observed in 3D virtual worlds. There is a growing focus on the communication about design
representations but the changes across the three design environments are not significant; (2) the highest percentage
of visual analysis for design development is observed in 3D virtual worlds. 3D modeling as the main design
approach in virtual worlds does not fit into the traditional analysis-synthesis model and should be studied further;
(3) As discussed, in virtual worlds designers use 3D models for both exploring design concepts and representing
final outcomes and often the transformation of the 3D models captures the design development process. The most
frequent operations on external representations are change-related activities in 3D virtual worlds, while create-
related activities occur most frequently in the two sketching environments.
Both similar and different patterns were observed when designers change from face to face collaboration to remote
collaboration, and from 2D sketching to 3D modeling. In a follow-up study, we further observed and surveyed the
13
9
th

same designer pairs experiences and preferences when they were given the choice to collaborate using the typical
3D modeling features supported in virtual worlds or the integrated remote sketching features. It was noted that (1)
different designers can have different preferences in applying 2D sketching, 3D modeling or the combination of the
two for design collaboration; (2) 3D models appear to be a more popular media for final design representations
compared to 2D sketches (3) both 2D sketches and 3D models can be used to explore abstract design concepts such
as spatial volumes and relationships as well as to develop external representations such as design details; and (4) 3D
virtual worlds support both collaborative and individual design tasks while remote sketching environments tend to
encourage collaborative tasks.
4. DESIGNING USING TANGIBLE USER INTERFACES
Until recently, research on TUIs focused on developing new systems and exploring technical options rather than
addressing users interaction experience in the new hybrid environments. This study explored the potentials and
perspectives of tangible interaction with an AR display for supporting design collaboration by measuring the effects
of TUIs on designers cognitive activities and design process. The aim was to gain a deeper understanding of the
designers experience of tangible interaction specifically, in terms of the collaborative affordance of TUIs, in the
context of co-design. In this study, physical manipulation and spatial interaction were considered as tangible
interaction. Physical manipulation of objects exploits intuitive human spatial skills, where movement and perception
are tightly coupled (Hornecker and Buur 2006; Sharlin et al. 2004). Spatial interaction is engaging in interaction
with the space created by the spatial arrangement of the design objects, and the use of this engagement for
communication in collaboration.
4.1 Experiments and coding scheme design
A tabletop system with TUIs including a horizontal surface and a vertical display has been developed at the Design
Lab at the University of Sydney to support problem solving, negotiation and establishing shared understanding in
collaborative design. As multiple, specialized tangible input devices for TUIs, 3D blocks with tracking markers in
ARToolKit (Billinghurst et al. 2001) can be attached to different functions, each independently accessible to 3D
virtual objects (Fitzmaurice 1996). The tabletop system was compared to a typical desktop system with GUIs for 3D
spatial planning tasks in a controlled laboratory experiment. Multiple designers can concurrently access the 3D
blocks whereas only one person can edit the model at a time using the mouse and keyboard. Comparing
conventional input devices such as mouse and keyboard, it was assumed that the affordances of the physical handles
of the TUIs facilitate two-handed interactions and thus offers significant benefits to collaborative working in design
collaboration (Granum et al. 2003; Moeslund et al. 2002)
The participants comprised three pairs of 2nd or 3rd year architecture students, with minimum of one years
experience as CAD users. Each pair performed both sessions, a TUI and a GUI, in one day for two different design
tasks. The chosen scenario was the redesign of a studio into a home office or a design office by configuring spatial
arrangements of furniture, where each 3D block can represent a piece of furniture in the tabletop system, and pre-
designed furniture can be imported from the library in ArchiCAD using a mouse and keyboard. Two design tasks
were developed to be similar in complexity and type, and the systems were assessed by letting designers discuss the
existing design and proposed new ideas in co-located collaboration. Designers conversation, interactions, and
gestures were videotaped while they designed the layout for four required areas according to the design
requirements and then the analysis of the data carried out using the protocol analysis method. The collected data in
the form of collaboration protocols were transcribed and then segmented using an utterance based technique. Each
utterance flagged the start of a new segment, and then for each segment relevant codes were assigned according to a
customized coding scheme.
The coding scheme comprised six categories at four levels: 3D modeling actions at the Action level, perceptual
activities at the Perceptual level, set-up goal activities and co-evolution at the Process level, and cognitive
synchronization and gesture actions at the Collaborative level. The Action level represents designers tangible
interaction with the external representation through the 3D modeling actions. The Perception level represents
designers perception of visuo-spatial features in the external representation. Designers perceptual activities in 3D
configurations are related to the reconsideration for different meaning and function of the same objects rather than
reconstructing unstructured images. The Process level represents designers problem-finding behaviors associated
with creative design. Specifically set-up goal activities refer to introducing new functional issues as new design
14
9
th

requirements. The co-evolution represents the exploration of two design spaces, problem and solution spaces. The
Collaborative level reflects designers collective cognition. Cognitive synchronization represents the argumentative
process in collaborative design, showing how designers construct a shared understanding of the problem and a
shared representation of the solution through negotiating activities. Gesture actions represent non-verbal design
communication in collaborative design. More information on the coding scheme can be found in Kims PhD thesis
(Kim 2006).
4.2 Protocol analysis results
TABLE 3 shows the mean values of segment durations in the design sessions ranging from 6.9 to 8.9 seconds. The
standard deviations are rather high (5.0 to 7.2 seconds), which suggests that the distribution of the segment durations
is widespread from the mean values. The average segment duration in TUI sessions (7.3 second) was shorter than
that in the GUI (8.3 second) sessions. Thus designers utterances during TUI sessions were on average shorter than
those in GUI sessions. Designers in the GUI session did not make as much progress in developing design solutions
as they did in the TUI sessions given the same amount of time.
TABLE. 3: Duration of segments
Pair 1 Pair 2 Pair 3
Session/Design task TUI
1/A
GUI
2/B
TUI
2/A
GUI
1/B
TUI
1/A
GUI
2/B

Task completion Yes Yes Yes No Yes No
Total time 1065 sec 1116 1206 sec 1196 sec 892 sec 911 sec
Segment no. 133 80 89 66 120 81
Mean (sec) 7.3 8.9 6.9 7.3 7.8 8.8
Std. Deviation 4.8 6.6 4.9 7.2 5.4 5.8
Session: 1 first session; 2 second session / Design task: A - Home office; B Design office
The difference in the task progression might also have affected the occurrence of cognitive actions as shown in
TABLE 4. With the direct, nave manipulability of physical objects and rapid visualization, designers in the TUI
session produced more multiple cognitive actions compared to designers in the GUI session (210 and 127). The
average occurrence of the perceptual actions in the TUI session (105) was twice that of the GUI session (57), and the
average occurrence of set-up goal actions in the TUI session (36) was almost twice that of the GUI session (20). In
order to identify how the different HCI environments influenced the proportion of the cognitive actions, the
relatively higher occurrences of each action category are shaded for each designer. Since we intended to identify the
trend in the differences between the two design sessions through the results of the collaborative study, we assumed
that a value higher than 2% between the two design sessions would indicate the changes according to the different
interaction modes. The trend shows that all designers produced more perceptual actions (50.0% and 44.9%) in the
TUI session, and more functional actions (32.9% and 39.4%) in the GUI session.
TABLE 4. Occurrence percentages of action categories
Perceptual Functional Set-up goal Total actions
TUI 136 (52.9%) 81 (31.5%) 40 (15.6%) 257 (100%)
Pair 1
GUI 57 (39.0%) 63 (43.2%) 26 (17.8%) 146 (100%)
TUI 100 (46.9%) 77 (36.2%) 36 (16.9 %) 213 (100%)
Pair 2
GUI 61 (45.2%) 53 (39.3%) 21 (15.5%) 135 (100%)
TUI 77 (49.4%) 54 (34.6%) 25 (16.0%) 156 (100%)
Pair 3
GUI 50 (47.6%) 39 (37.2%) 16 (15.2 %) 105 (100%)

TUI average 105 (50.0%) 69 (32.9%) 36 (17.1%) 210 (100%)
Pairs (average)
GUI average 57 (44.9%) 50 (39.4%) 20 (15.7%) 127 (100%)
With the focus being on designers spatial cognition, six cognitive action categories were investigated in terms of
four levels of the coding scheme. The encoded protocols were analyzed using a Mann-Whitney U test to examine
differences between the two sessions, and the structures of design behaviors were explored through the interactive
graphs. 3D modeling actions, perceptual and set-up goal activities were combined into generic activity components,
highlighting the different patterns of design behaviors. In order to compare in a same condition for two design
sessions, the total time of each GUI session was cut at a same time point as the corresponding TUI session.
15
9
th

Epistemic action refers to exploratory motor activity, where users would get the final stage through performing a
lot of physical actions even with no specific goal (Fitzmaurice and Buxto 1997; Kirsh and Maglio 1994). The
findings at the Action level suggested that in order to develop design ideas designers in the TUI session changed the
external representation by performing more movement modeling actions rather than compute it mentally. The short
and frequent 3D modeling actions are considered as epistemic actions, reducing their cognitive load and producing a
conversation style of interaction, to express and negotiate design ideas, establishing shared understanding in design
collaboration. These epistemic 3D modeling actions allowed designers to test their ideas quickly by proceeding in
small steps and then these continuities brought more opportunities to execute creative leaps in designing. The
findings at the Perception level imply that by manipulating 3D blocks, designers produced more perceptual actions,
whereby more new visuo-spatial features were created and discovered through performing these modeling actions.
The new perceptual information could have brought about new interpretations on the external representation since
thinking about emergent properties evokes shifting focus to a new topic. Furthermore, designers using TUIs
perceived more new and existing spatial relationships among elements, while focusing more on the existing
elements themselves in the GUI session. Perceiving more spatial relationships potentially encouraged designers to
explore related functional thoughts, to go beyond retrieving the visual information, and thus make abstract
inferences.
The findings at the Process level suggested that designers invented new design requirements by restructuring the
problem space based on the perceived information in a situated way rather than synthesizing design solutions fixed
on the initial requirements. The retrieval of knowledge, based on their expertise or past experience, for the new
constraints suggests that designers recall might be improved through the manipulation of 3D blocks. Furthermore,
they developed the formulation of the problem and alternatives for a solution throughout the design session, showing
a co-evolutionary process which is associated with creative outcomes in designing. Consequently, designers using
these 3D blocks would gain more opportunities to discover key concepts through the problem-finding process. The
embodied facilitation theme highlights that tangible interaction embodies structure that allows or hinders some
actions, and thereby shapes emerging group behavior. Spatial interaction refers to the fact that tangible interaction is
embedded in real space, thus has the potential to employ full-body interaction, acquiring communicative function.
The findings from the Collaborative level reveal that designers in the TUI session tend to establish more cognitive
synchronization through active negotiation processes, especially, the cycle of three codes Propose, Argument and
Resolution. The horizontal table and 3D blocks on the tabletop system might facilitate collaborative interactions by
working as the embodied structure. Spatial interaction with spaces was also facilitated while using 3D blocks, thus
designers in the TUI session produced more immersive gesture actions using hands and arms, leading to whole body
interaction with the external representation. Touch actions seemed to be beneficial for designers perceptual
activities because designers in the TUI sessions kept touching the 3D blocks, which might have simplified
designers mental computation through epistemic actions.
To sum up, the protocol analysis of the four levels of designers spatial cognition reveals that the epistemic 3D
modeling actions using TUIs put much less load on designers cognitive processes, thus resulting in the co-
generation of new conceptual thoughts and perceptual discoveries in the external representation. The off-loaded
designers cognition affected the design process by increasing problem-finding behaviors associated with creative
design and supported design communication, negotiation and shared understanding. In conclusion, the results of this
study suggest that tangible interaction afforded by TUIs provides important benefits for designers spatial cognition
and a cognitive and collaborative aid for supporting collaborative design. To verify these conclusions, more
designers need to be observed and their protocols analyzed.
5. IMPACT OF COLLABORATIVE TECHNOLOGIES ON DESIGN
In this paper we describe collaborative technologies as enablers of collective intelligence in design in two ways: (1)
the ability for collaboration to occur at the same time while the participants are remotely located and (2) the ability
to augment the perception of shared design drawings or models through new technologies for interacting with digital
models. Since the focus while designing is on the shared drawings or models of the design, critical aspects of
collaborative technologies for design are: the type of digital media available to represent the design, the interaction
technologies for creating, visualizing, and modifying the shared drawings or models, and the ways in which the
designers communicate and interact with each other.
16
9
th

In our study of 3D virtual worlds, we show how a multi-user virtual world that allows co-editing of 3D models
changes the behavior of designers in two important ways: given the same amount of time for the collaborative
session, the designers worked on the same task most of the time while collocated and only part of the time when
remotely located in physical space but collocated in the virtual world; and the 3D model as a focus for design
development and communication embodied concept and geometry in the 3D objects where this appears to be
separated in sketches. These results show that the type of digital media available, comparing 2D sketches and 3D
models, changes what designers talk about when they are collaborating; and the change from physically located
around a shared drawing to remotely located within a 3D model changes the working modes and encourages
designers to smoothly move between working on the same task and working on different aspects of the design.
In our study of TUIs compared to GUIs, we show how tangible interaction on a tabletop encourages the designers to
engage in more exploratory design actions, and more cognitive actions in general. We attribute this change to the
additional perceived affordances of the tangible blocks as interfaces to the digital model when compared to the
keyboard and mouse as the interface to the digital model. The blocks became specific parts of the model for the
designers as they moved the pieces on the tabletop, while the keyboard and mouse was negotiated to be different
parts of the model at different times. These affordances effectively allowed the designers to focus their actions
directly on the design alternatives and development rather than on the interface to the design model. This difference
affected the number of segments, an increased number when using TUIs implying an epistemic approach to
generating alternatives, and the number of cycles in the problem-finding process implying a potential for more
creative solutions.
The results from these two very different studies converge in a set of recommendations for the design of
collaborative technologies for designers: The development of multi-user 3D virtual worlds for designers, when
merged with current CAD capabilities, has the benefit of a smooth transition between working on the same task and
working on separate tasks. When using CAD systems for collaboration, a significant amount of time is spent
coordinating working on the same or different tasks at same or different times. A 3D virtual world enables a group
of designers to be more aware of each others presence when they are focused on designing. While the main role of
3D modeling in traditional CAD systems is design documentation, our study shows that 3D modeling can play
different roles in design collaboration from the early concept exploration to the final design representation. The
development of TUIs for 3D modeling changes the perception and interaction with digital models and should
become a standard alternative to the keyboard and mouse for design. Most collaborative technologies for designers
still rely on the keyboard and mouse for interacting with digital models. There is significant benefit and relatively
little training needed to incorporating TUIs into 3D modeling systems so that designers can choose the best
interaction style for the stage of the design and modeling tasks.
6. REFERENCES
Billinghurst, M., Kato, H., and Poupyrev, I. (2001). "MagicBook: a Transitional AR Interface." Computer Graph 25,
745-753.
Brave, S., Ishii, H., and Dahley, A. (1999). "Tangible Interface for Remote Collaboration and Communication." CHI
'99: Conference on Human Factors in Computing Systems, Pittsburgh, Pennsylvania, USA, 394-401.
Cross, N., Christiaans, H and Dorst, K (1996). Analysing Design Activity, John Wiley & Sons Ltd, Chichester, UK.
Cross, N., and Cross, A. (1996). "Observations of Teamwork and Social Processes in Design." Analysing Design
Activity, N. Cross, H. Christiaans, and K. Dorst, eds., John Wiley & Sons Ltd, Chichester, UK.
Deitz, P., and Leigh, D. (2001). "DiamondTouch: A Multi-User Touch Technology." UIST 2001: The 14th Annual
ACM Symposium on User Interface Software and Technology, Orlando, Florida, USA.
Dorst, K., and Dijkhuis, J. (1995). "Comparing Paradigms for Describing Design Activity." Design Studies 16(2),
261-275.
Ericsson, K., and Simon, H. (1993). Protocol Analysis: Verbal Reports as Data, MIT Press, Cambridge,
Massachusetts, USA.
Fitzmaurice, G. (1996). "Graspable User Interfaces", University of Toronto, Toronto.
17
9
th

Fitzmaurice, G., and Buxto, W. (1997). "An Empirical Evaluation of Graspable User Interfaces: Towards
Specialized, Space-multiplexed Input." CHI '97: Conference on Human Factors in Computing Systems, Los
Angeles, USA, 43-50.
Fjeld, M., Bichsel, M., and Rauterberg, M. (1998). "BUILD-IT: an Intuitive Design Tool based on Direct Object
Manipulation." Gesture Workshop '96: Gesture and Sign Language in Human-Computer Interaction,
Bielefeld, Germany, 297-308.
Gabriel, G., and Maher, M. (2002). "Coding and Modeling Communication in Architectural Collaborative Design."
Automation in Construction 11, 199-211.
Gero, J., and Mc Neill, T. (1997). "An Approach to the Analysis of Design Protocols." Design Studies 19(1), 21-61.
Goldschmidt, G. (1996). "The Designer as a Team of One." Analysing Design Activity, N. Cross, H. Christiaans,
and K. Dorst, eds., John Wiley & Sons, Chichester, UK.
Granum, E., Moeslund, T., and Strring, M. (2003). "Facilitating the Presence of Users and 3D Models by the
Augmented Round Table." Presence 2003: The 6th Annual International Workshop on Presence 19.
Ishii, H., and Ullmer, B. (1997). "Tangible Bits: Towards Seamless Interfaces between People, Bits and Atoms."
CHI '97: Conference on Human Factors in Computing Systems, Los Angeles, USA, 234-241.
Kim, M. (2006). "Effects of Tangible User Interfaces on Designers' Spatial Cognition," University of Sydney,
Sydney.
Kirsh, D., and Maglio, P. (1994). "On Distinguishing Epistemic from Pragmatic Action." Cognitive Science 18, 513-
549.
Lloyd, P., Lawson, B., and Scott, P. (1995). "Can Concurrent Verbalization Reveal Design Cognition?" Design
Studies 16 (2), 237-259.
Maher, M., Rosenman, M., Merrick, K., and Macindoe, O. (2006a). "DesignWorld: An Augmented 3D Virtual
World for Multidisciplinary Collaborative Design." CAADRIA 2006, Osaka, Japan, 133-142.
Maher, M. L., Bilda, Z., Gu, N., Gul, F., Huang, Y., Kim, M. J., Marchant, D., and Namprempree, K. (2006b).
"CRC Design World Report." University of Sydney, Sydney.
Moeslund, T., Strring, M., and Liu, Y. (2002). "Towards Natural, Intuitive and Non-Intrusive HCI Devices for
Roundtable Meetings." MU3I: Workshop on Multi-User and Ubiquitous User Interfaces, Funchal, Madeira,
Portugal, 25-29.
Norman, D., and Draper, S. (1986). "User Centered System Design: New Perspectives on Human-Computer
Interaction." Lawrence Erlbaum Associates, Hillsdale, NJ.
Schn, D. (1983). The Reflective Practitioner: How Professionals Think in Action, Basic Books, New York.
Sharlin, E., Watson, B., Kitamura, Y., Kishino, F., and Itoh, Y. (2004). "On Tangible User Interfaces, Humans and
Spatiality." Personal and Ubiquitous Computing 8(5), 338-346.
Simon, H. (1992). The Sciences of the Artificial, The MIT Press, Cambridge, Massachusetts, USA.
Singhal, S., and Zyda, M. (1999). Networked Virtual Environments: Design and Implementation, ACM Press, New
York.
Stempfle, J., and Badke-Schaub, P. (2002). "Thinking in Design Teams - an Analysis of Team Communication."
Design Studies 23(5), 473-496.
Suwa, M., Purcell, T., and Gero, J. (1998). "Macroscopic Analysis of Design Processes based on A Scheme for
Coding Designers' Cognitive Actions." Design Studies 19(4), 455-483.
Suwa, M., and Tversky, B. (1997). "What Do Architects and Students Perceive in Their Design Sketches?" Design
Studies 18(4), 385-403.
Valkenburg, R., and Dorst, K. (1998). "The Reflective Practice of Design Teams." Design Studies 19(3), 249-271.
18
9
th

A NOVEL CAMERA-BASED SYSTEM FOR COLLABORATIVE
INTERACTION WITH MULTI-DIMENSIONAL DATA MODELS
*

Michele Van den Bergh,
Computer Vision Laboratory, ETH Zurich, Zurich, Switzerland;
vamichae@vision.ee.ethz.ch
Jan Halatsch,
Chair of Information Architecture, ETH Zurich, Zurich, Switzerland;
halatsch@arch.ethz.ch
Antje Kunze,
kunze@arch.ethz.ch
Frdric Bosch, PhD,
bosche@vision.ee.ethz.ch
Luc Van Gool, Prof.,
vangool@vision.ee.ethz.ch
Gerhard Schmitt, Prof.,
schmitt@ia.arch.ethz.ch
ABSTRACT: In this paper, we address the problem of effective visualization of and interaction with multiple and
multi-dimensional data supporting communication between project stakeholders in an information cave. More
exactly, our goal is to enable multiple users to interact with multiple screens from any location in an information
cave. We present here our latest advancements in developing a novel human-computer interaction system that is
specifically targeted towards room setups with physically spread sets of screens. Our system consists of a set of
video cameras overseeing the room, and of which the signals are processed in real-time to detect and track the
participants, their poses and hand-gestures. The system is fed with camera based gesture recognition. Early
experiments have been conducted in the Value Lab, which has been introduced recently at ETH Zurich, and they
focus on enabling the interaction with large urban 3D models being developed for the design and simulation of
future cities. For the moment, experiments consider only the interaction of a single user with multiple layers (points
of view) of a large city model displayed on multiple screens. The results demonstrate the huge potential of the
system, and the principle of vision based interaction for such environments. The work continues on the extension of
the system to a multi-user level.
KEYWORDS: Information cave, interaction, vision, camera, hand gestures.
1. INTRODUCTION
1.1 Product Information Models for Design and Simulations
Future cities, standing for evolving medium-size and mega-cities, have to be understood as a dynamic system a
network that bridges different scales, such as local, regional, and global scales. Since such a network comprises
several dimensions, for example social, cultural, and economic dimensions it is necessary to connect active research,

*This work was supported by the Competence Center for Digital Design Modeling (DDM) at ETH Zurich.

19
9
th

project management, urban planning as well as communication with the public to establish a mutual vision, or to
map the desires of the involved participants.
In the last few decades, the use of computers, software and digital models has expanded within many fields related
to the Architecture, Engineering, Construction and Facility Management (AECFM), where Facility may refer to
commercial, industrial or infrastructure building assets, and also cities. However, it is only recently that researchers
have started tackling the problems of the compartmentalization of this expansion within these different fields
corresponding to the multiple stakeholders of such project. And, this expansion occurred without wider project
integration. For example, in urban planning, multiple different digital models are often used to perform different
analyses such as: CO
2
emissions, energy consumption and traffic load. Nonetheless, significant progresses have
recently been made in the integration of information models into what are now commonly referred to Building
Information Models (BIM), City Information Models (CIM), etc.
These integrated models enable earlier and more systematic (sometimes automated) detection of conflicts different
multiple analysis and processes. However, the resolution of these conflicts still requires human negotiations, and
effective methods and technologies for interacting collaboratively with the information in order to resolve detected
conflicts are still missing. The main complexity here is that large projects, such as large scale planning projects,
require the involvement of many technical experts and other stakeholders (e.g. owners, pubic) who approach
projects from many different view points, which results in many different types of conflicts that must resolve
collaboratively.
In order to address this problem, holistic participative planning paradigms (governing process management, content
creation as well as design evaluation) have to evolve, and consider new software and hardware solutions that will
enable the different stakeholders to effectively work collaboratively.
1.2 Example: Dbendorf Urban Planning Project
Todays urban planning and urban design rely mainly on static representations (e.g. key visuals, 3D models). Since
the planning context and its data (for example scenario simulations) are dynamic, visual representations need to be
dynamic and interactive too, resulting in the need for physical environments enabling such dynamic processes.
During spring semester 2009 students researched how to establish design proposals in a more collaborative manner.
The focus was on an urban planning project, the rehabilitation of the abandoned Swiss military airport in Dbendorf.
The main goal of this research project was to develop an interactive shape grammar model (Mller, 2009), which
was implemented with the CityEngine (http://www.procedural.com/cityengine). In combination with real-time
visualization using Autodesk Showcase (http://usa.autodesk.com/adsk/servlet/index?id=6848305&siteID=123112),
a better understanding design interventions was achieved.
While this research project showed the feasibility of collaborative interactive design, the experiments, then
conducted in the Value Lab (see section 2) showed that the interactivity offered by such information caves did not
always meet the expectations of the users (see analysis in section 2).

FIG. 1: As a result of collaborative city design workshops a new use for an abandoned military airport in the out-
skirts of Zurich had been implemented with the collaborative interaction tools that are available at the Value Lab.
20
9
th

1.3 Information Visualization Caves
Information visualization caves have been investigated in order to enable stakeholders to sit in a single room and
collaboratively solve conflicts, during planning, construction or operation. Such caves are typically designed with
complex multimedia settings to enable participants to visualize the project model, and the possible conflicts at hand,
from multiple points of view simultaneously (e.g. owner vs. user vs. contractor, contractor A vs. contractor B).
Traditional human-computer interaction devices (e.g. mouse, keyboard) are typically focused to fulfill single user
requirements, and are not adapted to work with the multiplicity of participants and the multi-dimensionality (as well
as multiplicity) of the data sets representing large projects. Solutions have however been proposed to improve
interactivity. A multi-screen setup can drastically enhance collaboration and participatory processes by keeping
information present to all attendees, and such setup is common in information caves (Gross et al., 2003, Knig et al.,
2007). Additionally, (multi-) touch screens are now available as more intuitive multi-user human-computer
interaction devices. However, despite their definite advantages for interactions with multiple users, particularly in
table settings, multi-touch screens remain inadequate for use in rooms with physically spread sets of screens, as they
require the users to constantly move from a screen to the other.
2. VALUE LAB
The ETH Value Lab (see figure 2) is a special kind of information visualization room, and was designed as a
research platform to guide and visualize long-term planning processes while intensifying the focus on the
optimization of buildings and infrastructures through new concepts, new technologies and new social behaviors to
cut down CO
2
emissions, energy consumption, traffic load, and to increase the quality of life in urban environments
(Halatsch and Kunze, 2007). It helps researchers and planners to combine existing realities with planned
propositions, and overcome the multiplicity (GIS, BIM, CAD) and multi-dimensionality of the data sets representing
urban environments (Halatsch et al., 2008a and 2008b).
The Value Lab consists of a physical space with state-of-the art hardware (supercomputer), software (e.g. urban
simulation and CAD/BIM/GIS data visualization packages) and intuitive human-computer interaction devices. The
interface consists of several high-resolution large area displays including:
Five large screens with a total of 16 mega pixels and equipped with touch interface capabilities; and
two; and
Three FullHD projectors. Two projectors form a concatenated high-resolution projection display with
4 Megapixel in resolution. That particular configuration is for example used for real-time landscape
visualization. The third projector delivers associated views for videoconferencing, presentation and
screen sharing.
The computing resources, display and interaction system produces a tremendous amount of possible configurations
especially in combination with the connected computing resources. The system manages all computing resources,
operation systems, displays, inputs, storage and backup functionality in the background as well as lighting
conditions and different ad hoc user modes.
As a result, The Value Lab forms the basis for knowledge discovery and representation of potential transformations
of the urban environment, using time-based scenario planning techniques in order to test the impact of varying
parameters on the constitution of cities. It shows how the combination of concepts for hardware, software and
interaction can help to manage digital assets and simulation feedback as well as promoting visual insights from
urban planners to associated stakeholders in a human-friendly computer environment (Fox, 2000).
However, as discussed earlier, we found out that beside the direct on-screen manipulation of information, a
technology was needed to steer larger moderated audiences inside a project, and that offers a more integrated
navigation and usability behavior as well as permitting a wider overview on the main contents presented.
Therefore we are investigating a novel touch-less interaction system with camera-based gesture recognition. This
system is presented below and early experimental results are presented in section 4.
21
9
th

FIG. 2: The Value Lab represents the interface to advanced city simulation techniques and acts as the front-end of
the ETH Simulation Platform.
3. VISION SYSTEM
In this section, we describe the vision system, that detects and tracks hand gestures of a user in front of a camera
mounted on top of a screen as shown in figure 3. The goal of the system is to enable the interaction of the person
with a 3D model.
A recent review of vision-based hand pose estimation (Erol et al., 2007) states that currently, the only technology
that satisfies the advanced requirements of hand-based input for human computer interaction is glove-based sensing.
In this paper, however, we aim to provide hand-based input without the requirement of such markers.
The first contribution is an improved skin color segmentation algorithm that combines an offline and an online
model. The online skin model is updated at run-time based on color information taken from the face region of the
user. This skin color segmentation is used to detect the location of the hands. The second contribution is a novel
hand gesture recognition system, which combines the classification performance of average neighborhood margin
maximization (ANMM) with the speed of 2D Haarlets. The system is example-based, matching the observations to
predefined gestures stored in a database. The resulting system is real-time and does not require the use of special
gloves or markers.

FIG. 3: Person interacting with a camera and screen.
3.1 Skin Color Segmentation
The hands of the user are located using skin color segmentation. The system is hybrid, combining two skin color
segmentation methods. The first is a histogram-based method, which can be trained online, while the system is
running. The advantage of this system is that it can be adapted in real-time to changes in illumination and to the
person using the system. The second method is trained in advance with a Gaussian mixture model (GMM). The
22
9
th

benefit of the offline trained system is that it can be trained with much more training data and is more robust.
Howe`ver, it is not robust to changes in illumination or to changes in the user.
3.1.1 Online model
Every color can be represented as a point in a color space. A recent study (Schmugge et al., 2007) tested different
color spaces, and concluded that the HSI (hue, saturation and intensity) color space provides the highest
performance for a three dimensional color space, in combination with a histogram-based classifier.
A nice characteristic is that the histograms can be updated online, while the system is running. Two histograms are
kept, one for the skin pixel color distribution (H
skin
), and one for the non-skin colors (H
non-skin
). For each frame in the
incoming video stream, the face region is found using a face detector such as the one in OpenCV
(http://opencvlibrary.sourceforge.net/), and the pixels inside the face region are used to update H
skin
. Then, the skin
color detection algorithm is run and it finds the face regions as well as other skin regions such as the hands and
arms. The pixels that are not classied as skin are then used to update H
non-skin
.
3.1.2 Offline model
In the GMM-based approach, the pixels are transformed to the rg color space. A GMM is fitted to the distribution of
the training skin color pixels using the expectation maximization algorithm as described in (Jedynak et al., 2002).
Based on the GMM, the probabilities P(skin|color) can be computed offline, and stored in a lookup table.
3.1.3 Post processing
On one hand, the histogram-based method performs rather well at detecting the skin color pixels under varying
lighting conditions. However, as it bases its classification on very little input data, it has a lot of false positives. On
the other hand, the GMM-based method performs well in constrained lighting conditions. Under varying lighting
conditions it tends to falsely detect white and beige regions in the background. By combining the results of the
histogram-based and the GMM-based methods, many false positives can be eliminated. The resulting segmentation
is improved further in additional post processing steps, which include median filtering and connected components
analysis.
3.2 Hand Gesture Recognition
The hand gesture recognition algorithm is based on the full body pose recognition system using 2D Haarlets
described in (Van den Bergh et al., 2009). Instead of using silhouettes of a person as input for the classifier, hand
images are used.
3.2.1 Classifier input
The hands are located using the skin color segmentation algorithm described in section 4.1. A cropped grayscale
image of the hand is extracted, as well as a segmented silhouette, which are then concatenated into one input sample,
as shown in figure 4. The benefit of using the cropped image without segmentation, as shown on the right, is that it
is very robust for noisy segmentations. Using the silhouette based on skin color segmentation only, as shown on the
left, the background influence is eliminated. Using the concatenation of both gives us the benefit of both input
sample options.

FIG. 4: Example of an input sample.
3.2.2 Haarlet-based classifier
For details about the classifier we refer to (Van den Bergh et al., 2009). It is based on an average neighborhood
margin maximization (ANMM) transformation T, which projects the input samples to a lower dimensional space, as
23
9
th

shown in figure 5. This transformation is approximated using Haarlets to improve the speed of the system. Using
nearest neighbors search, the coefficients are then matched to hand gestures stored in a database.

FIG. 5: Structure of the classifier illustrating the tranformation T (dotted box), approximated using Haarlets. The
Haarlet coefficients are computed on the input sample. The approximated coefficients (that would result from T) are
computed as a linear combination C of the Haarlet coefficients.
4. EXPERIMENTS
In this section, we describe the demo application that allows for the visualization of 3D models that can be loaded
into the program. Using hand gestures, the user can zoom in on the model, pan and rotate it.
4.1 Gestures
The hand gesture classifier is trained based on a set of training samples containing the gestures shown in figure 6.
An example of the system detecting these static gestures is shown in figure 7.

FIG. 6: The gestures that are trained in the hand gesture classifier.

FIG. 7: Examples of the hand gesture recognition system detecting different hand gestures.
The hand gesture interaction in this application is composed of the hand gestures shown in figure 6. It recognizes the
gestures and movements of both hands to enable the manipulation of the object/model. Pointing with one hand
selects the model to start manipulating it. By making two fists, the user can grab and rotate the model along the z-
axis. By making a fist with just one hand, the user can pan through the model. By making a pointing gesture with
both hands, and pulling the hands apart, the user can zoom in and out of the model. The open hands release the
model and nothing happens until the user makes a new gesture. An overview of these gestures is shown in figure 8.
24
9
th

(a) select (start interaction)

(b) rotating

(c) panning

(d) zooming

(e) release (do nothing)
FIG. 8: The hand gestures used for the manipulation of the 3D object on the screen.
The time delay of the vision system on average is 31 ms. This is the time recorded from sending the command to
grab the image from the camera, to sending the resulting interaction commands to the user interface. This allows us
to run the system at a refresh rate of 30 Hz. The accuracy of the recognition system (using the three hand poses
shown in figure 6) is 99.8%. Increasing the number of trained poses from three to ten results in a recognition
accuracy of 98.2%. The accuracy of the hand localization based on the skin color segmentation is less than a pixel,
granted that the hand is segmented correctly. The cases where the hand is not segmented are: when the hand
overlaps with the face of the user, or overlaps with a similarly colored person or object in the background. These are
predictable and could be eliminated in future work with a form of depth estimation, of which unfortunately no
accurate real-time implementations exist to our knowledge at time of writing.
4.2 Application
The interaction system above has been implemented as an extension of an open-source 3D model viewer, the GLC
Player (http://www.glc-player.net/). This enables us to: (1) load models in multiple formats (OBJ, 3DS, STL, and
OFF) and of different sizes, and (2) use our hand interaction system in combination with standard mouse and
keyboard interaction. Pressing a button in the toolbar activates the hand interaction mode, after which the user can
start gesturing to navigate through the model. Pressing the button again deactivates the hand interaction model and
returns to the standard mouse-keyboard interaction mode.
We conducted experiments by installing our system in the Value Lab and tested with multiple 3D models, and in
particular with a model created as part of the Dbendorf urban planning project. This model represents an area of
about 0.6 km2 and is constituted of about 4000 objects (buildings, street elements, trees) with a total of about
500,000 polygons. Despite this size, our system achieved frame rates of about 30fps (frame per second), which is
sufficient for smooth interaction. Examples of the user zooming, panning and rotating through the 3D model are
shown in figures 9, 10 and 11 respectively. In each figure, the left column shows side and back views of the system
in operation at the beginning of the gesture, and the right columns the same views but at the end of the gesture.
The hand interaction mode is currently only available for model navigation (rotation, panning and zooming), all the
other features of the viewer being only accessible in mouse-keyboard interaction mode. Nonetheless, our
implementation enables simple extensions of the hand interaction mode. In the near future, we for instance aim to
enable the hand interaction mode for object selection (to view its properties).
25
9
th

FIG. 9: Zooming into the model.

FIG. 10: Panning the model.
26
9
th

FIG. 11: Rotating the model.
5. CONCLUSION
In this paper, we first described the need for novel human-computer interaction tools, enabling users in information
visualization caves to simultaneously interact with large amounts of information displayed on multiple screens
spread around the cave. Todays urban design tasks could be significantly enhanced in terms of interaction
especially when different stakeholders are involved. Currently available interaction devices, such as mouse-
keyboard or screen (multi-) touch capabilities, are often not adapted to such requirements, and this was confirmed in
an urban design project conducted in the Value Lab at ETH Zurich.
A novel solution for human-computer interaction was then introduced that is based on vision. Compared to currently
existing systems, it presents the advantage of being marker-less. Experiments, conducted in the Value Lab,
investigated the usability of this system in a situation as realistic as possible. For these, our interaction system has
been integrated to a 3D model viewer, and tested with a large 3D model of an urban development project. The
results show that our system enables a stable, smooth and natural interaction with 3D models at refresh rates of 30
Hz.
Nonetheless, these results remain preliminary. The system is not always as robust as it should be, and its
applicability to enable multiple users to simultaneously interact with multiple screens remains to be demonstrated.
Future work will thus be targeted to: (1) extend the set of viewing features accessible through hand gesture (in
particular object selection and de-selection); (2) further improve the robustness of the system, particularly with
respect to different users; and (3) develop a larger system containing multiple cameras and enabling the interaction
of multiple users with different screens.
6. REFERENCES
Erol, A., Bebis, G., Nicolescu, M., Boyle, R. D., and Twombly, X. (2007). Vision-based hand pose estimation: a
review. Computer Vision and Image Understanding, vol. 108, 52-73.
27
9
th

Fox, A., Johanson, B., Hanrahan, P., and Winograd, T. (2000). Integrating information appliances into an
interactive workspace. IEEE Computer Graphics & Applications, vol. 20, no. 3, 54-65.
Gross, M., Wrmlin, S., Naef, M., Lamboray, E., Spagno, C., Kunz, A., Koller-Meier, E., Svoboda, T., Van Gool,
L., Lang, S., Strehlke, K., Moere, AV., and Staadt, O. (2003). Blue-c: a spatially immersive display and 3D
video portal for telepresence. ACM Transactions on Graphics, 819-827.
Halatsch, J., and Kunze, A. (2007). Value Lab: Collaboration In Space. IV 2007: 11th International Conference
Information Visualization, Zurich, Switzerland, July 4-6, 376-381.
Halatsch, J., Kunze, A., Burkhard, R., and Schmitt, G. (2008a). ETH Value Lab - A Framework For Managing
Large-Scale Urban Projects. 7th China Urban Housing Conference, Chongqing, China, Sept. 26-27.
Halatsch, J., Kunze, A., and Schmitt, G. (2008b). Using Shape Grammars for Master Planning. DCC 2009: 3rd
Conference on Design Computing and Cognition, Atlanta, Sept. 21-26, 655-673.
Jedynak, B., Zheng, H., Daoudi, M., and Barret, D. (2002). Maximum entropy models for skin detection. ICVGIP
2002: 3rd Indian Conference on Computer Vision, Graphics and Image Processing, Ahmadabad, India, Dec.
16-18, 276-281.
Knig, W. A., Bieg, H.-J., Schmidt, T., and Reiterer, H. (2007). Position-independent interaction for large
highresolution displays. IHCI 2007: IADIS International Conference on Interfaces and Human Computer
Interaction, Lisbon, Portugal, July 6-8, 117-125.
Mller, P., Wonka, P., Haegler, S., Ulmer, A., and Van Gool, L. (2006). Procedural Modeling of Buildings. ACM
Transactions on Graphics, vol. 25, no. 3, 614-623.
Schmugge, S. J., Jayaram, S., Shin, M. C., and Tsap, L. V. (2007). Objective evaluation of approaches of skin
detection using ROC analysis. Computer Vision and Image Understanding, vol. 108, 4151.
Van den Bergh, M., Koller-Meier, E., and Van Gool, L. (2009). Real-time body pose recognition using 2D or 3D
Haarlets. International Journal on Computer Vision, vol. 83, 72-84.
28
9
th

TOWARDS A COLLABORATIVE ENVIRONMENT FOR SIMULATION
BASED DESIGN
Michele Fumarola, Ph.D. Candidate,
Systems Engineering Group
Faculty of Technology, Policy and Management, Delft University of Technology
Jaffalaan 5, 2628 BX Delft, The Netherlands;
Tel. +31 (0)15 27 89567
m.fumarola@tudelft.nl
http://www.tudelft.nl/mfumarola
Stephan Lukosch, Assistant Professor,
Tel. +31 (0)15 27 83403
s.g.lukosch@tudelft.nl
http://www.tudelft.nl/sglukosch
Mamadou Seck, Assistant Professor,
Tel. +31 (0)15 27 83709
m.d.seck@tudelft.nl
http://www.tudelft.nl/mseck
Cornelis Versteegt, Senior Project Manager,
APM Terminals Management BV
Anna van Saksenlaan 71, 2593 HW The Hague, The Netherlands;
cornelis.versteegt@apmterminals.com
ABSTRACT: Designing complex systems is a collaborative process wherein modeling and simulation can be used
for support. Designing complex systems consists of several phases; specification, conceptual and detailed design
and evaluation. Modeling and simulation is currently mostly used in the evaluation phase. The goals, objectives and
IT support for each phase differ. Furthermore, multi-disciplinary teams are involved in the design process. We aim
at providing an integrated collaborative environment for modeling and simulation throughout entire design
projects. The proposed architecture, called Virtual Design Environment, consists of three main components: a
design, a visualization, and a simulation component. The layout of the design is made in the design component. The
design component has been developed as an AutoCAD plug-in. This approach was chosen, due to AutoCAD being
used in many complex design projects. The AutoCAD plug-in communicates the design decisions to the simulation
component. The processes that will take place once the system is built, are simulated by the simulation component.
Finally, the results of the simulation are sent to the visualization component. The visualization component provides
an interactive 3D environment of the design and can serve decision makers as a tool for communication, evaluation
and reflection. In this paper, we present the architecture of this environment and show some preliminary results.
KEYWORDS: Collaborative design, modeling and simulation, virtual environment, knowledge sharing, complex
systems
1. INTRODUCTION
As introduced by Simon (1977), decision making is composed of structuring the problem, evaluating alternatives
upon criteria and selecting the best alternative. Modeling and simulation (M&S) is often seen as a tool to analyze
29
9
th

(Zeigler & Praehofer, 2000) and is therefore mainly used in the evaluation phase of a decision making process. We
propose that M&S can be introduced much earlier in the design process of complex systems; more precisely in the
phase of structuring the problem. Many different design methodologies can be found in literature for the design of
complex systems (INCOSE, 2009; Pielage, 2005; Sage & Armstrong, 2000). There are many similarities in all of
these design methodologies. We do not focus on an individual design methodology, but more on the similar phases
in the methodologies. We identify the following phases; specification, conceptual and detailed design, and
evaluation. Simulation can add value in each of these mentioned phases. Currently, however, simulation is mostly
used in the evaluation phase. When using simulation earlier in the design, the designers and decision makers ability
to generate alternatives will be enhanced. The designers can generate more alternatives and study them more
comprehensively. However, designing a complex system is commonly a collaborative process wherein multiple
actors are involved. These actors have varying interests and fields of expertise. To achieve a fruitful process, these
actors must first acquire a shared understanding of the problem domain and afterwards be able to collaborate
effectively on this problem.
To design a complex system, support is therefore needed from different perspectives. An integrated environment to
support the design of a complex system should not only be able to support design, but also simulation. As a shared
understanding among all involved actors (Piirainen, Kolfschoten, & Lukosch, 2009) is a major challenge in
collaborative design, the design and the simulation results need to be visualized to present them to the various actors
active in the design process. Moreover, these actors need to able to collaborate on the design by simulating and
visualizing the result. These perspectives are shown in Figure 1. We therefore propose to use a collaborative design
environment, based on M&S which supports designers in all the design phases mentioned earlier.
FIG. 1 The design environment should cover different perspectives.
In this paper, we will present how such an environment can be achieved. We will begin by describing a case study
we conducted at a large container operator which is currently dealing with the design of automated container
terminals. Designing a container terminal is a complex problem and automation brings additional challenges as it is
a novel approach wherein little experience has been gathered so far. From this problem, we will gather requirements
which will be used to design the environment. After discussing the related work on this topic, we will present our
approach for such an environment. Subsequently, some preliminary experiences will be presented based on
interviews with domain experts. We will finally conclude the paper and present our future work.
2. REQUIREMENTS ANALYSIS
Starting from an exploratory case study performed at a container terminal operator, we identify requirements for the
design environment that we propose. We will first present the case study and focus on a common approach on
designing a complex system as for instance a container terminal. From there, we will perform an analysis and extract
general requirements.
30
9
th

2.1 Case study
Automated container terminals are gaining momentum as the advantages in terms of costs and productivity are
getting clear. In this novel type of terminals, operations (e.g. picking up a container, placing it on a vessel, etc) are
computer operated, which has a number of advantages: lower life cycle costs, improvement in terms of safety,
reduction of damage to containers, higher service levels, etc. Designing these terminals is however a complex
endeavor: a large number of actors are involved in the design process and the design space (i.e. the number of
alternative designs) is quite large. Moreover, experience in the design of automated container terminals is still
limited as the number of these terminals around the globe is small. During the design process, two types of situations
may occur: brainstorming sessions and reflection.
In the brainstorming sessions, the different people involved in the process, gather in the same location and work
together on the design of the new terminal. In these situations, a white board is mostly used to sketch the design and
to present ideas. Sketching the design is however mostly coarse, and often the design is quickly described in words
e.g. stating the number of needed equipment without specifying the location. Supporting documents are shared
between the participants, notes are usually taken on paper or typed on a computer, and there is verbal
communication.
In the reflection, the actors involved in the design, work separately to concentrate on their particular task: e.g. the
CAD designer will make an initial drawing of the terminal, and the business analysts will use their business model.
There is little interaction between these actors until they reach a certain goal for which a new meeting will be
scheduled.
2.2 Analysis
The brainstorming sessions are mainly paper based and unstructured. As certain expertise is needed to use CAD
environments such as AutoCAD, these tools are seldom used in such sessions. Although CAD designers are skilled
in these environments, the remaining actors are not. These environments do however offer the opportunity of
making exact initial designs without running into misunderstandings. The first requirement is therefore the
possibility of sharing a CAD environment without exposing non-experts with unneeded complexity. Thereby, non-
experts have the possibility to gain a better understanding of the current design which is one of the major challenges
in complex collaborative design projects (Piirainen, et al., 2009).
In these design, understanding the dynamics of the system can be a hard task. As the system comprises a large
amount of entities, the exploration of alternatives and the experimentation with these alternatives needs to be
supported. Inputting a decision in the design environment should therefore be facilitated by taking into account such
things as contrasting decisions, physical feasibility, future outcomes, etc. The link between the design environment
and a possible simulation environment is hereby made.
The decision making process is supported by a large amount of documentation which is mainly printed or shared
through a computer network. Querying for a specific item of information can be challenging and inefficient. This is
due to the lack of structure of the common approach of sharing documentation. Having the ability to easily find the
right document to make a decision is important. This is however only possible if the right information is available in
the decision making environment: having the possibility to easily share documentation is therefore required.
For enabling collaborative decision making, a collaborative environment has to support a number of functionalities.
At the core of a collaborative decision-making process, actors have to reach a shared understanding of the problem
domain. For that purpose, the actors need to communicate their understanding to the others actors and they need the
possibility to discuss the feasibility of their decisions. The latter also requires understanding of recent developments
and changes in the design. As a result, the collaborative environment has to offer various means for synchronous as
well as asynchronous communication and mechanisms for achieving group awareness (Gutwin, Greenberg,
Roseman, & Sasse, 1996).
From this analysis, a number of requirements can be extracted:
1. Visualization for non-experts of the system under investigation
2. Documentation sharing and structuring
31
9
th

3. Simulation-based experiments
4. Communication and awareness support for the participants
3. RELATED WORK
Modeling and simulation has been used often in the analysis of container terminal design. Stahlbock & Voss (2008)
reviewed the existing literature on container terminal logistics and found simulation as a tool to analyze different
aspects of container terminals. Nevertheless there is little reference to the use of M&S during the synthesis phase of
the design process. Ottjes et al. (2006) discuss their solution using three main functions to make the conceptual
design of a container terminal. This environment however, mainly focuses on simulation, leaving out the
collaborative aspects. More attention on collaboration in simulation projects for container terminal logistics have
been put by Versteegt, Vermeulen, & van Duin (2003), although existing tools were used which do not offer explicit
support for collaboration.
On the design of complex systems, more work has been done which comes closer to fulfilling our stated
requirements. Peak, et al. (2007) presented their approach based on SysML where they use components to design a
system, they do not however concentrate on collaboration during the design. Paredis et al. (2001) also introduced a
component based simulation environment for the design of mechatronic systems. Their solution enables multiple
users to collaborate on the design of such a system, but do not take into account the different skills of the actors
involved, as their scenario assumes users with comparable backgrounds. Comparable environments exist for virtual
prototyping in specialized engineering (e.g. automobile, aeronautical) industries.
On collaborative design in virtual environments, various examples are at hand. Shiratuddin & Breland (2008)
present an environment for architectural design that uses a 3D game engine to present the final design. They argue
shared understanding is achieved across interdisciplinary groups. Rosenman, Smith, Maher, Ding, & Marchant
(2007) discuss a solution that has different views (CAD and 3D virtual environment) on a given design in the AEC
domain. However, simulation is out of the scope of this research. Further examples of collaborative design in virtual
environments can be found in Conti, Ucelli, & Petric (2002), and Pappas, Karabatsou, Mavrikios, & Chryssolouris
(2006).
The conclusions found in Sinha, Lian, Paredis, & Khosla (2001) suggest a lack of close integration of design and
modeling & simulation tools. They also recommend the use of design repositories that would provide a way to share
knowledge about the system that is being designed. Furthermore, the few integrated environments do not support
collaboration across interdisciplinary groups.
4. APPROACH
Based on the requirements set in section 2, we will present our approach which comprises 4 parts: visualization,
sharing, simulation, and collaboration. These parts come forward from the different requirements. Firstly, we
discussed the need of non-experts to be able to understand the complex designs made in environments such as
AutoCAD. In order to do so, an understandable way of visualizing such a design is needed. Secondly, we identified
the requirement of sharing documents which are important for the decision making process. Thirdly,
experimentation with a given design is desirable, for which simulation is needed to predict the workings of the
system. Lastly, communication between the different actors needs to be supported as well as collaboration while
working on the design.. With these parts, an architecture can be finally constructed, which we will discuss as well.
4.1 Visualization
During the design process, visualization plays an important role in order to understand the problem at hand. In the
case of container terminals, it becomes even more important as the point of focus is a physical facility.
The design of a container terminal is commonly done in a CAD environment such as AutoCAD. This offers the
designers the possibility to precisely specify the design and later to use the drawings for the construction process.
CAD drawings are usually rather complex, making them hard to use by less proficient users such as business
analysts. An alternative way of visualizing the future facility is therefore required.
32
9
th

Various studies have been performed showing 3D as the preferred approach on visualizing complex physical
systems to increase interest and understanding of a user and to facilitate dialogue (Fabri, et al., 2008; Terrington,
Napier, Howard, Ford, & Hatton, 2008). It is therefore desirable to have a translation from the CAD drawings to a
3D environment understandable by the actors involved in the design process. According to Whyte et al. (2000) this
translation can be achieved in several ways: by the library approach, the simple translation and the database
approach. The library approach uses a set of predefined 3D models to map with the CAD drawing. On the other
hand, the simple translation purely transforms the CAD drawing to a 3D model, using CAD drawings which are
drawn with 3D vectors. Lastly, the database approach uses a central database with a description of an object from
which a 3D model and a CAD drawing can be extracted.
Once this translation took place, the visualization is in place to be used. The designers interact with the 3D
environment to input their design decisions and to support their communication on ideas and possible issues. This
relieves them from having to describe everything in words, as what happens traditionally.
4.2 Sharing
The design process of a complex system is commonly accompanied by a large amount of documentation. A lack of
structure and inability to look through this documentation slows down the design process. Furthermore, everyone
involved in the design process should have access to the available documentation. Because of this, sharing and
structuring documentation should be part of the design environment.
The structuring of documentation can exploit the structure of the actual system. An ontology of the system would
provide the entities to which the documentation can be linked. In the case of a container terminal, the structure of the
system would be the physical structure of the terminal: e.g. yard, stacks, and different types of equipment. If a user
would want to find out information about a specific entity, the logical place to look for would be the visualized
entity in the virtual environment.
Using the structure, sharing can also become a possibility. By providing the functionality of adding documentation
by choosing a specific entity, this can be achieved. Furthermore, additional information such as notes, ideas and
issues can be shared as well using the same functionality.
4.3 Simulation
When a system is composed of many parts interacting dynamically in a non-trivial way, it becomes difficult to
understand or predict their performance through mere static analysis. In this context, simulation is a powerful
instrument for enabling experimentation with the system-to-be, in order to anticipate the implications of the design
decisions at a relatively low cost. Simulation is routinely used in this way to evaluate design alternatives through
what-if analysis.
For most clients involved in a modeling and simulation effort, only the experimentation results (and their
interpretations) have added value. All preceding phases, although appraised as essential, are typically seen as
technical work and are naturally externalized. This situation stems from the fact that advanced modelling and
simulation requires specialized training and is not at the core of the clients business. This state of affairs produces a
gap between the client and the constructor of the simulation. On the one hand, the constructors only input is a
system description document that could never capture the richness of the design activity. On the other hand, the
client loses the benefits that would have accrued from using simulation earlier in the cycle to explore a larger design
space.
A collaborative environment integrating design and simulation would bridge the proverbial gap, allowing the (CAD)
designs to be translated into simulations models by an underlying simulation environment and thus facilitating
experimentation within the design activity, fostering creativity and reactivity. There is of course no magic bullet.
Such capability will only be possible if a comprehensive library of domain specific models has been constituted and
individually validated beforehand. An ontology guarantying the compositional soundness of the design is also an
essential asset. This ontology can be developed using the System Entity Structure formalism (Zeigler & Hammonds,
2007). The use of simulation formalisms supporting modularity and hierarchical construction of models is a strong
requirement of the simulation part. A system theoretic formalism such as DEVS is particularly suited for this goal
because due to its property of closed under coupling.
33
9
th

4.4 Communication and awareness support
The major requirements for enabling collaborative decision-making are related to achieve a shared understanding by
means of communication and awareness support. According to Gerosa, Fuks, Raposo, & Lucena (2004) cooperating
users must be able to communicate and to coordinate themselves. When communicating, users might generate
commitments and define tasks that must be completed to accomplish the common group goal. These tasks must be
coordinated so that they are accomplished in the correct order and at the correct time with respect to possible
external restrictions. To accomplish these tasks the users have to cooperate in a shared environment. However, while
cooperating, unexpected situations might emerge that demand new communication. In such communication new
commitments and tasks might be defined, which again must be coordinated to be accomplished in cooperation. In
this cooperation cycle, awareness plays a central role. Every user action that is performed during communication,
coordination, or cooperation generates information. Some of this information involves two or even more users, and
should be made available to all cooperating users so that they can become aware of each other. This helps to mediate
further communication, coordination, and cooperation and build up a shared understanding of their common group
goals and to synchronize their cooperation.
Considering the above, we decided to link the different components of the system architecture by means of
communication functionality and awareness widgets. In the following, we briefly describe the functionality we want
to integrate by means of patterns
1
for computer-mediated interaction (Schmmer & Lukosch, 2007) which capture
best practices in the design of collaborative environments. For awareness purposes, we want to integrate a USER
LIST, an INTERACTIVE USER INFO, a CHANGE INDICATOR, a TELEPOINTER, a REMOTE FIELD OF VISION. The USER
LIST will show which decision-makers are currently present and take a look a the design visualization. The
INTERACTIVE USER INFO will be coupled with the USER LIST allow decision-makers to directly select between
different communication possibilities when choosing one decision-maker from the USER LIST. We will use the
CHANGE INDICATOR pattern within the visualization environment to highlight changes which a decision-maker has
not seen yet and thereby make decision-makers aware of the recent changes of the designer. The TELEPOINTER will
allow the stakeholders in the visualization environment to point to specific design parts and thereby support an
ongoing discussion. Finally, the REMOTE FIELD OF VISION will allow decision-makers to identify in which parts of
the simulation the others are interested in. Thereby, REMOTE FIELD OF VISION will foster discussion among the
decision-makers as well as raise the level of shared understanding.
To further increase the shared understanding, we decided to add functionality for synchronous as well as
asynchronous communication. We want to integrate an EMBEDDED CHAT, a FORUM, SHARED ANNOTATIONS as well
as a FEEDBACK LOOP. The EMBEDDED CHAT will be available in the visualization environment. It will allow
decision-makers to directly communicate with each other and discuss general questions concerning the design. We
will also include a FORUM in which decision-makers can start asynchronous discussions. By using THREADED
DISCUSSIONS these FORUMS can also serve a knowledge base and repository for the VDE. To allow decision-makers
a artifact-centred discussion we will support SHARED ANNOTATIONS. Decision-makers will be able to add
annotations to specific points of interest within the visualization environment and share these annotations with the
other decision-makers. Furthermore, these SHARED ANNOTATIONS will be pushed to the design environment so that
the designer becomes aware of questions as well as discussions concerning the design of the complex environment.
Thereby, we implement a FEEDBACK LOOP. In addition, the designer will be able to comment on the annotations so
that a THREADED DISCUSSION can evolve within a SHARED ANNOTATION.
5. SOLUTION
5.1 Architecture
The different parts which result from the requirement, serve as a basis to construct a software architecture for the
proposed environment. Each part contains different components in order to achieve the required functionality. A
component diagram of this architecture is shown in Figure 2.
The visualization part contains a 3D visualizer and 2D textual output. The 3D visualizer renders the 3D environment
which is based on the CAD drawing. The visualization is currently being handled in a rendering engine, namely

1
Please note that pattern names are set in SMALL CAPS.
34
9
th

Ogre3D. This enables us to accurately render the large amount of objects commonly found in container terminals.
The 2D textual input provides the possibility to visualize textual information as well as pictures, videos, etc. This is
needed for documentation sharing and some functionality to enable collaboration.
The environment is fed by different sources. First of all, a database can feed the documentation to the environment
and information can also be added by users who wish to share it. On the other hand, simulation results can be used
as input by the simulation data feed.
The user interface gives access to the virtual environment with its documentation and simulation results. This is the
main interface for the decision makers involved in the design process. The world loader reads the CAD drawing
which is being edited by the designers. Once the designers finish to work on the layout, the CAD drawing is send to
the 3D virtual environment which will visualize the new design. This design will hereafter serve to run the
simulation and thereby study the performance of the new design. Feedback from non-designers can hereafter be fed
back to the CAD environment making it possible for the designers to handle new requests. The interaction between
designers and non-designers is supported as outlined in the previous section. This process can run multiple times
throughout the design process. The process is sketched in Figure 3.
5.2 Discussion
The environment presented here provides the means to collaboratively design a system using simulation across a
multidisciplinary group of actors. To achieve this, we presented an architecture based on the different parts resulting
from the requirements. This provides us an environment wherein a complete design process can take place. In
contrast to existing environments (for instance the ones discussed in the related work-section), the presented
environment provides the possibility to design a complex system collaboratively, visualize the design to achieve
shared understanding and support the design by sharing existing knowledge. The strength of the environment is
therefore not found in the individual components, which are already known and widespread, but in the integrated
environment wherein the components reside.
The design of the system is supported by simulation which is used to evaluate viable alternatives. Actors can achieve
insight into the workings of the systems which has not been physically developed yet. The actors, which have
different backgrounds, use a view on the system which they can understand (2D CAD or 3D) and can share existing
and new knowledge on the given system. Lastly, communication between these actors is possible through the
environment.
FIG. 2 An architecture for the proposed environment.

35
9
th

6. EXPERIENCES
Initial feedback from designers and decisions makers that have worked with the Virtual Design Environment is very
positive. First, the Virtual Design Environment provides more insight by using photorealistic 3D images and movies
of high quality. The designers used 2D images and movies in the past. The users especially appreciated the realism
of the environment. The Virtual Design Environment uses models and CAD drawing to create a realistic
representation. Schematic representations were used in the past. Secondly, the Virtual Design Environment is an
environment in which the users can freely move around. Traditionally, the movies contained predefined flight paths
that offered no flexibility in the point of view of the designers. The designers and decision makers indicated that
they see high value in using the Virtual Design Environment. They expect a number of benefits that were not
anticipated by the developers. The Virtual Design Environment can be used as a training tool before the terminal is
implemented. Operators can be trained for their future job using the Virtual Design Environment. The operators can
be trained in a safe environment without disrupting day-to-day operations. The decision makers also expect that the
Virtual Design Environment will have high value for the commercial aspects of the terminal. The Virtual Design
Environment can be shown to customers and used as a selling tool. Finally, the Virtual Design Environment can
be used for communicating the design to port authorities, governments and other stakeholders. The design can be
presented in an understandable format to all stakeholders.
7. CONCLUSIONS
In this paper we presented our work towards the Virtual Design Environment, an environment meant to support the
design of complex systems in a multi-actor setting. From an exploratory case study at a large container terminal
operator, we gathered the requirements for the design environment. These requirements were marked as five
components: visualization, sharing, communication, collaboration, and simulation. Although the design environment
is still under research, preliminary results could be gathered. Consultation with domain experts showed that the
design environment can indeed result in a more effective and efficient design process. Nevertheless, extended
evaluations have to be done to confirm these early expectations. Future work will therefore consist of further
development of the environment. Moreover, generalizing the environment should be considered, instead of
restraining it to the design of automated container terminal.
8. REFERENCES
Conti, G., Ucelli, G., & Petric, J. (2002). JCAD-VR: a collaborative design tool for architects. Paper presented at the
4th International Conference on Collaborative virtual environments.
Fabri, D., Falsetti, C., Iezzi, A., Ramazzotti, S., Rita Viola, S., & Leo, T. (2008). Virtual and Augmented Reality. In
H. H. Adelsberger, Kinshuk, J. M. Pawlowski & D. G. Sampson (Eds.), Handbook on Information
Technologies for Education and Training (pp. 113-132): Springer Berlin Heidelberg.
FIG. 3 Design process using the Virtual Design Environment
36
9
th

Gerosa, M. A., Fuks, H., Raposo, A. B., & Lucena, C. J. P. (2004). Awareness Support in The AulaNet Learning
Environment. Paper presented at the International Conference on Web-based Education.
Gutwin, C., Greenberg, S., Roseman, M., & Sasse, M. (1996). Workspace Awareness in Real-Time Distributed
Groupware: Framework, Widgets, and Evaluation. Paper presented at the People and Computers XI
(Proceedings of the HCI'96).
INCOSE (2009). The International Council on Systems Engineering Retrieved July 13th, 2009, from
http://www.incose.org
Ottjes, J. A., Veeke, H. P. M., Duinkerken, M. B., Rijsenbrij, J. C., & Lodewijks, G. (2006). Simulation of a
multiterminal system for container handling. OR Spectrum, 28(4), 447-468.
Pappas, M., Karabatsou, V., Mavrikios, D., & Chryssolouris, G. (2006). Development of a web-based collaboration
platform for manufacturing product and process design evaluation using virtual reality techniques.
International Journal of Computer Integrated Manufacturing, 19(8), 805-814.
Paredis, C. J. J., Diaz-Calderon, A., Sinha, R., & Khosla, P. K. (2001). Composable Models for Simulation-Based
Design. Engineering with Computers, 17(2), 112-128.
Peak, R. S., Burkhart, R. M., Friedenthal, S. A., Wilson, M. W., Bajaj, M., & Kim, I. (2007). Simulation-Based
Design Using SysML - Part 2: Celebrating Diversity by Example. Paper presented at the INCOSE
International Symposium.
Pielage, B.-J. (2005). Conceptual Design of Automated Freight Transport Systems: Methodology and Practice. Delft
University of Technology, Delft.
Piirainen, K., Kolfschoten, G., & Lukosch, S. (2009). Unraveling Challenges in Collaborative Design: A Literature
Study. Paper presented at the 15th Collaboration Researchers' International Workshop on Groupware.
Rosenman, M. A., Smith, G., Maher, M. L., Ding, L., & Marchant, D. (2007). Multidisciplinary collaborative design
in virtual environments. Automation in Construction, 16(1), 37-44.
Sage, A., & Armstrong, J. J. (2000). Introduction to systems engineering. New York, NY, USA: John Wiley & Sons
Inc.
Schmmer, T., & Lukosch, S. (2007). Patterns for Computer-Mediated Interaction: John Wiley & Sons.
Shiratuddin, M. F., & Breland, J. (2008). Development of a Collaborative Design Tool for Virtual Environment
(CDT-VE) Utilizing a 3D Game Engine. Paper presented at the 8th International Conference on Construction
Applications of Virtual Reality 2008.
Simon, H. (1977). The new science of management decision. Upper Saddle River, NJ, USA: Prentice Hall PTR.
Sinha, R., Lian, V. C., Paredis, C. J. J., & Khosla, P. K. (2001). Modeling and Simulation Methods for Design of
Engineering Systems. Journal of Computing and Information Science in Engineering, 1(1), 84-91.
Stahlbock, R., & Voss, S. (2008). Operations research at container terminals: a literature update. OR Spectrum,
30(1), 1-52.
Terrington, R., Napier, B., Howard, A., Ford, J., & Hatton, W. (2008). Why 3D? The Need For Solution Based
Modeling In A National Geoscience Organization. Paper presented at the 4th International Conference on
GIS in Geology and Earth Sciences.
Versteegt, C., Vermeulen, S., & van Duin, E. (2003). Joint Simulation Modeling to Support Strategic Decision-
Making Processes. Paper presented at the 15th European Simulation Symposium and Exhibition.
Whyte, J., Bouchlaghem, N., Thorpe, A., & McCaffer, R. (2000). From CAD to virtual reality: modelling
approaches, data exchange and interactive 3D building design tools Automation in Construction, 10(1), 43-
55.
Zeigler, B. P., & Hammonds, P. E. (2007). Modeling & Simulation-Based Data Engineering: Introducing
Pragmatics into Ontologies for Net-Centric Information Exchange: Academic Press.
37
9
th

Zeigler, B. P., & Praehofer, H. (2000). Theory of modeling and simulation: Academic Press.

38
9
th

EMPIRICAL STUDY FOR TESTING EFFECTS OF VR 3D SKETCHING
ON DESIGNERS COGNITIVE ACTIVITIES
Farzad Pour Rahimian, Dr.,
Department of Architecture, Faculty of Design and Architecture, Universiti Putra Malaysia;
ulud_rah@yahoo.com
Rahinah Ibrahim, Associate Professor Dr.,
Department of Architecture, Faculty of Design and Architecture, Universiti Putra Malaysia;
rahinah@putra.upm.edu.my
ABSTRACT: To optimise the level of information integration of the critical conceptual architectural-engineering
design stage, designers need to employ more flexible and intuitive digital design tools during this phase. We studied
the feasiblity of using 3D sketching in VR in order to replace current non-intuitive Computer Aided Design (CAD)
tools that designers would rather not using during the conceptual architectural process. Using the capablities of
VR-based haptic devices, the proposed 3D sketching design interface relies on the sense of touch for simplifing
designing and integrating designers cognitions and actions in order to improve design creativity. Adopting a
cognitive approch to designing, the study compares the effectivenss of the proposed VR-based design interface with
common manual sketching design interfaces. For this purpose, we conducted a two-session expeiment which
comprises of design activites of three pairs of 5th year architecture students. In comparing the designers collective
cognitive and collaboratibve actions the study employs design protocol analisys research methodology. This study
evaluated the designers spatial cognition at four different levels: physical-action, perceptual-actions, functional-
actions, conceptual-actions. The results show that compared to the traditional design interfaces, the utilized VR-
based simple and tangible interface improved designers cognitive design activities. We claim that due to the
capability of reversing any undesired changes, 3D sketching design interface increases designers motivation and
courage for performing more cognitive activities than conventional approach. Increasing the occurrence frequency
of designers perceptual actions, the 3D sketching interface associated cognition with action and supported the
designers epistemic actions which are expected to increase design creativity. The rich graphical interface in 3D
sketching system has led to the occurrence of more unexpected discoveries and situative inventions that carried
both problem and solution spaces towards maturity. Moreover, the increment in the percentage of new physical
action has decreased the amount of unnecessary physical actions and possibility for shifting from pragmatic actions
towards epistemic actions. Results of this study can help the development of cutting-edge information technologies
in either design or education of architecture. They also can help in the creation of training programs for
professional graduates who are competent in multidisciplinary teamwork and equally competent in utilizing IT/ICT
in delivering their building projects within time and budget.
KEYWORDS: Conceptual Design, 3D Sketching, Multidisciplinary, Virtual Reality, Protocol Analysis.
1. INTRODUCTION
Early conceptual phases of the design process are characterized by fuzziness, coarse structures and elements, and a
trial-and-error process. Craft and Cairns (2006) mentioned that searching for form and shape is the designers
principal goal. During these stages the chances of correcting errors are the highest and the use of low-expenditure
sketches and physical models is crucial. Cross (2007) believes that the thinking processes of the designer hinge
around the relationship between internal mental processes and their external expression and representation in
sketches. Cross (2007, p.33) is confident that the designer has to have a medium which enables half formed ideas to
be expressed and to be reflected upon: to be considered, revised, developed, rejected and returned to.
Using current CAD tools has bad effects on designers reasoning procedures and hampers their cognitive activities
(Ibrahim and Pour Rahimian In review). Existing literature (e.g. Bilda and Demirkan, 2003) have also shown that
due to some inherent characteristics of current CAD tools, designers are not quite successful when they are working
with such digital design tools during conceptual design phase. However, literature recommends to designers to
migrate from manual design tools to digital design systems in order to integrate the whole design process (Kwon et
al. 2005). Literature also highlights that the intangible and arduous user interface of current CAD systems are two
39
9
th

major issues which hamper designers creativity during conceptual design phase (Kwon et al. 2005). Consequently,
this paper proposes VR 3D sketching which is a simple and tangible VR-based design interface as an alternative
solution to replace ordinary CAD systems during conceptual design phase. Regenbrecht and Donath (1996) posit
that 3D sketching in VR can be used as instantaneous reflection and feedback of the design procedure in which the
user can act with the digital design support tool in a spontaneous, game-like, empirical manner. Consequently, VR
can offer the ideal interface for free artistic visualization and linking creative experimentation and accurate
manufacturing-oriented modelling (Fiorentino et al. 2002).
(Regenbrecht and Donath 1996)
This paper reports a conducted empirical experiment for reaffirming efficiency of the proposed system in conceptual
architectural design phase. The paper presents the results of a comparison on design activities between a VR-based
simple and tangible interface and a traditional pen and paper sketching interface. It focuses on designers collective
cognitive activities when working on similar design tasks. Here the traditional sketching method is selected as a
baseline to be compared to a proposed 3D sketching design methodology. The purpose is to reveal the cognitive
impacts of the proposed design system. Five pairs of 5
th
year architecture students experienced with the traditional
design and CAD systems were selected as participants for this experiment. During the experiment, protocol analysis
methodology (Ericsson and Simon 1993; Lloyd et al. 1995; Schn 1983a) was selected as a research and data
acquisition method to explore the effects of the different media on designers spatial cognition.
2. VR 3D SKETCHING AND COGNITIVE APPROACH TO DESIGNING
Goldschmidt and Porter (2004) defined designing as a cognitive activity which entails the production of sequential
representations of a mental and physical artefact. Tversky (2005) believes that constructing the external or internal
representations, designers are engaged in spatial cognition process in which the representations serve as cognitive
aids to memory and information processing. Schn (1992) asserted that with execution of action and reflection, each
level of representation makes designers evolve in their interpretations and ideas for design solutions. Such cognitive
approach to designing considers design media as something beyond mere presentation tools. In this approach,
reflections which are caused by design media are expected to either stimulate or hamper designers creativity during
design reasoning.
2.1 Creativity
The term, creative, is usually used as a value of a design artefact (Kim and Maher 2008). Yet, according to Visser
(2004) in cognitive psychology discussions this is linked to design activity which also comprises particular
procedures that have the potential to produce creative artefacts. Cross and Dorst (1999) define the creative design
procedure as a sort of non-routine designing activities that usually is differentiated from the others by the appearance
of considerable events or unanticipated novel artefacts.
Situative-inventions is a more evolved model for measuring design creativity. According to Suwa et al. (2000),
situated-invention of new design requirements (S-invention) can be considered as a key for inventing a creative
artefact. Based on this model, when introducing the new constraints for design artefact, designers capture significant
parts of the design problem and go beyond a synthesis of solutions that suits the given requirements. On the other
hand, Cross and Dorst (1999) posit the modelling of the design creativity as a co-evolution of problem and goal
spaces. Co-evolutionary design is an approach to problem-solving (Kim and Maher 2008). In this approach the
design requirements and design artefacts are formed disjointedly while mutually affecting each other. Kim and
Maher (2008) believe that in this approach the changing of a problem causes some changes in the designers insight
of a problem situation.
Unexpected discoveries (Suwa et al. 2000) is a key for evaluating creative design process. They define it as
perceptual activities of articulating tacit design semantics into visuo-spatial forms in an unanticipated way for later
inspection. They found that the appearance of unexpected discoveries of visuo-spatial forms and S-invention are
strongly related to each other. As another approach, Suwa and Tversky (2001) in a constructive approach posit that
co-evolution of new conceptual semantics and perceptual discoveries improve designers understandings of
external representations. In Gero and Damskis (1997) opinion, constructive perceptions allow designers to change
their focus and to understand design problem in a different way in which re-interpretation may be stimulated so that
designers find the opportunity to be more creative. (Suwa and Tversky 2001)
40
9
th

2.2 Spatial cognition using tangible user interfaces (TUIs) in VR 3D sketching
Tangible user interfaces (TUIs) is a technology which comprises of digital information and physical objects to
virtually mimic an absolute environment. According to Kim and Maher (2008), as opposed to a simple time-
multiplexed technology which is used in ordinary input devices (e.g. a mouse), the main advantages of TUIs is the
space multiplexing input technology which is able to control various functions at different times. One instance of
such machinery is haptic technology. In computer science discussions, the term haptic relates to the sense of touch.
In other words, this technology is a technology which unites the user to a digital system by simulating the sense of
touch and applying force-feedback, vibrations, and motions to the user (Basque Research 2007). We believe that this
physical interaction with real world is the quality that Stricker et al. (2001) describe as the technology which
augments our cognition and interaction in the physical world. Based on its capabilities we believe that haptic
technology provides an advanced TUI for designers.
As mentioned above, in haptic technology the sense of touch is not limited to a feeling and it facilitates a real-time
interactivity with virtual objects. According to Brewster (2001), haptic technology is a huge pace in VR area since it
allows users to utilize their touch sense to feel virtual objects. He argues that although touch is an extremely
powerful sense, it has so far been abandoned in the digital world. In this research we focused on the role of force-
feedback facilitated by SensAble Technology TUIs in forming a designers spatial cognition. (Fitzmaurice 1996)
Fitzmaurice (1996) relates the effects of such interfaces to the quality of motor activities. He uses definitions of
epistemic or pragmatic actions (Kirsh and Maglio 1994) to classify designers motor activities. In his definition,
epistemic actions are taken to reveal hidden information or that are difficult for mankind to compute mentally. He
believes that the physical activities help people perform easier, faster and more reliable on internal cognitive
computation. This is something like using the fingers when counting. According to Fitzmaurice (1996) the epistemic
actions can improve cognition by: 1) decreasing the involvement of memory in mental computation (space
complexity), 2) decreasing the number of mental computation steps (time complexity), and 3) decreasing the rate of
mental computation error (unreliability). On the other hand, Fitzmaurice (1996) define pragmatic actions as physical
actions which primary physically perform to make the user closer to the aim.
Fitzmaurice (1996) believes that such activities can strongly help an integrated human cognitive model in which
necessary information for each step can be provided by both mental processing resources and physical modifying.
He argues that consequently this can support perceiving external environment. Fitzmaurice (1996) concludes these
arguments positing that mental modules can trigger motor activity which propose to lead to changes in the physical
environment that assists cognitive processes. He argues that when using physical objects, users are able to
manipulate and influence their environment. This paper seeks the effects of proposed VR 3D sketching interface of
designers collective cognitive activities based on above mentioned cognitive approach to designing.
3. COMPARING VR 3D SKETCHING AND TRADITIONAL SKETCHING
This section presents the developed methodological framework besides the details of the conducted experimental
protocol analysis for testing proposed VR 3D sketching interface. It comprises explanations of five-step conducted
protocol analysis methodology which is proposed by van Someren et al. (1994). The five steps are as follow: 1)
conducting experiments, 2) transcribing protocols, 3) segmentation procedure, 4) developing coding scheme and
encoding protocol data besides developing research hypotheses, and 5) selecting strategies to analyze and interpret
the encoded protocols. The section ends with the explanation of the adopted strategies for validation and reliability.
3.1 Experimental protocol analysis for testing VR 3D sketching design interface
As discussed above, this study proposes a simple and tangible VR-based design system as an alternative solution to
replace ordinary CAD systems during conceptual design phase. This section reports an empirical experiment for
reaffirming efficiency of the proposed system in conceptual architectural design phase. The study proposes a
comparison on design activities between a VR-based simple and tangible interface and a traditional pen and paper
sketching interface. It focuses on designers collective cognitive activities when working on similar design tasks.
Here the traditional sketching method is selected as a baseline to be compared to a proposed 3D sketching design
methodology. The purpose is to reveal the cognitive impacts of the proposed design system. Five pairs of 5
th
year
architecture students experienced with the traditional design and CAD systems were selected as participants for this
41
9
th

experiment. Each pair was required to complete two design tasks which utilized traditional and 3D sketching design
media sequentially. During the experiment, protocol analysis methodology (Ericsson and Simon 1993; Lloyd et al.
1995; Schn 1983a) was selected as a research and data acquisition method to explore the effects of the different
media on designers spatial cognition.
3.1.1 Development of research instrument, SensAble haptic devices vs. manual sketching
In order to compare the impacts of the proposed 3D sketching design methodology on the designers cognitive
activities, we proposed a simple traditional conceptual design package as a baseline system and a VR-based digital
design package as a 3D sketching environment. The utilized traditional conceptual design package comprises design
pencils and pens, butter papers, and simple mock-up materials e.g. polystyrene as well as drafting tables and chairs.
On the other hand, the proposed VR-based digital design package consists of a tablet PC for supporting designers
preliminary ideations and a desktop PC for supporting digital design process. Both systems are shown in Figure 1.

FIG.1: Prepared traditional (left) and 3D sketching (right) design settings
The commercial software Adobe Photoshop
TM
was installed on the tablet PC to provide the layering ability that was
available in the traditional sketching system which used butter papers. Designers therefore were able to produce
preliminary sketches directly on the screen of the tablet PC. The utilized desktop PC comprised of a monitor as the
output system and a keyboard, a mouse, and a 6DF SensAble haptic device which supported force-feedback and
vibration as the input system. During the experiment we used an evaluation version of ClayTools
TM
software as the
basic environment for the modelling and spatial reasoning. ClayTools
TM
is VR-based software which is designed for
being used with SensAble haptic devices. The experiment expects the used 6DF coordination system to solve the
previous 2D mouse coordination problems which designers faced in the traditional CAD systems. Ultimately, the 3D
sketching interface was set up to offer the expected simplicity and tangibility of using VR in design.
3.1.2 Design tasks
In order to test the effects of the interface on all aspects of conceptual design, the designers were required to perform
in two comprehensive conceptual design sessions for full three hours each. Therefore, during these sessions,
designers were asked to undergo all stages of conceptual design: initial bubble diagramming, developing design
idea, and preparing initial drawings. The goal of the first design task was to design a shopping centre with maximum
200000 square feet built area. On the other hand, the goal of the second design task was to design a culture and art
centre with maximum 150000 square feet built area. In order to make designers concentrate on design itself rather
than presentation, during both sessions they were required not to use more than one colour in the presentations.
3.1.3 Experimental set-ups: traditional session vs. 3D sketching session
We started our experiment with 5 pairs of designers. However, since 2 groups failed to complete their training
sessions, we performed the experiment for only 3 groups of designers. The traditional sessions were held at a design
studio while the 3D sketching sessions were held in an office which was being used as a VR lab during the
experiment. During both sessions to record all of the events during the design sessions, two digital cameras and one
sound recorder were used. The purpose of the first camera was to record all the drawings which were produced
during the test. The other camera was set up to record the designers design gestures and behaviours. Finally, a
digital sound recorder was used to record the designers conversations for transcription purposes. The designers
were asked to sit on one side of the table which was facing both cameras. Without interfering with the designers
42
9
th

thinking process, the experimenter was present at the design studio and Lab to prevent any technical problem. The
explained setting is shown in Figure 2 and Figure 3.

FIG.2: Experimental set-up for traditional sessions

FIG.3: Experimental set-up for 3D sketching sessions (Ericsson and Simon 1993)
3.2 Protocol analysis
Due to a tendency towards the objective ways for studying designers problem-solving processes, protocol analysis
is the emerging prevailing method for design studies (Cross et al. 1996). Kim and Maher (2008) have advocated
using this methodology for analyzing and measuring designers' cognitive actions instead of using subjective self-
reports such as questionnaires and comments. Having all strategies of protocol analysis methodology, since our
study focuses on designers cognitive activities and also the use of concurrent method is impossible for collaborative
works (Kan 2008), the retrospective content-oriented protocol analysis is selected as the data collection strategy for
our research. Herewith, the designers worked naturally while the entire processes were recorded. After finishing
their sessions, the designers were required to transcribe their sessions using the aid of the recorded media. Their
transcriptions as well as the recorded media provided the research data.
3.2.1 Unit of analysis and strategy in parsing the segments
Since this research focuses on designers cognitive actions during both traditional and digital sessions and the tested
hypotheses are relying on their actions, the codes assigned to the different segments are considered as our units of
analysis. Besides, we followed Ericsson and Simons (1993) suggestions in segmenting the process depending on
the occurrences of the processes.
3.2.2 Coding Scheme
Basically, the coding scheme which is used in this study is borrowed and adopted from Suwa et al.s (1998, 2000)
studies. The developed coding scheme comprises main categories from the Suwa et al.s (1998, 2000) coding
scheme and our own sub-categories based on the designers different actions which were particular for our study.
Ultimately, the proposed coding scheme characterizes designers spatial cognition at four different levels: physical-
action, perceptual-actions, functional-actions, and conceptual-actions. Although it is not claimed that our sub-
categories are the best possible answers for this kind of study, we are confident that this coding scheme is capable to
embrace all cognitive codes that our designers produced during the experiment. Details are shown in Table 1.

43
9
th

TABLE. 1: Developed coding scheme for 4 action-categories and their sub-categories
Category ID Index Description
Physical P DirectlyrelatedtothePactions
Dactions Da Depictingactionswhichcreateordealwithanyvisualexternalrepresentationofdesign
CreateNew Dacn New Tocreateanewdesignelementorasymbol(drawingcircles,lines,textures,arrows,etc)
ModifyExisting Dame New Toedittheshape,size,textureetcofthedepictedelement
CreateMask Dacm Old Tocreateamaskareaforselectingsomething
RelocateExisting Dare New Tochangethelocationortheorientationofthedepictedelement
CopyExisting Dace New Toduplicateanexistingelement(fordigitalworkonly)
TracingExisting Date Old Totraceovertheexistingdrawing
RemoVeExisting Dave New Toremoveanexistingobjector(fordigitalworkonly)toundoanycommandortoturnoff
Lactions La Lookactionswhichincludeinspectingapreviousdepictionsoranygiveninformation
InspectBrief Laib Old Referringtothedesignbrief
TurnonObject Lato Old Turningontheinvisibleobjects
InspectScreen Lais Old Lookingatscreen(fordigitalworkonly)
InspectsHeet Laih Old Lookingatdesignsheet(formanualworkonly)
Inspect3DModel Lai3 Old Lookingatvirtualorphysical3Dmodelwhilerotatingit
Mactions Ma OtherPactionswhichcanfallintothemotoractivities
MovePen Mamp New Tomovepenonthepaperorboardwithoutdrawinganything
MoveElement Mame New Tomoveanelementinthespacearbitrarilyforfindingnewspatialrelationship
TouchModel Matm New Totoucheitherphysicalorvirtualmodeltostimulatemotoractivities
ThinkingGesture Matg New Anyarbitrarilygesturewhichmotivatesthinkingaboutdesign
Perceptual Pe
Actions related to the paying attention to the visuospatial features of designed elements or
space
Pvisual Pv Discoveryofvisualfeatures(geometricalorphysicalattributes)oftheobjectsandthespaces
NewVisual Pnv Unexp.D Newattentiontoaphysicalattributesofanexistingobjectoraspace(shape,sizeortexture)
EditVisual Pev Other Editingoroverdrawingofanelementtodefineanewphysicalattribute
NewLocation Pnl Unexp.D Newattentiontothelocationofanelementoraspace
EditLocation Pel Other
Editing or overdrawing of the location of an element or a space to define a new physical
attribute
Prelation Pr Discoveryofspatialororganizationalrelationsamongobjectsorspaces
NewRelation Pnr Unexp.D Newattentiontoaspatialororganizationalrelationsamongobjectsorspaces
EditRelation Per Other Editingoroverdrawingofaspatialororganizationalrelationsamongobjectsorspaces
Pimplicit Pi Discoveryofimplicitspacesexistinginbetweenobjectsorspaces
NewImplicit Pni Unexp.D Creatinganewspaceorobjectinbetweentheexistingobjects
EditImplicit Pei Other
Editingtheimplicitspaceorobjectinbetweentheexistingobjectsbyeditingorrelocatingthe
objects
Functional F
Associating visual or spatial attributes or relations of the elements or the spaces with
meanings,etc
Finteractions Fi Interactionsbetweendesignedelementsorspacesandpeople
NewInteractive Fni Associatingainteractivefunctionwithajustcreatedelementorspaceoraspatialrelation
ExistingInteractive Fei Associatingainteractivefunctionwithanexistingelementorspaceoraspatialrelation
ConsiderationInteractive Fci
Thinking of an interactive function to be implemented independently of visual features in the
scene
Fpsychological Fp Peoplespsychophysicalorpsychologicalinteractionswithdesignedelementsorspaces
NewPsychological Fnp Associatingapsychologicalfunctionwithajustcreatedelementorspaceoraspatialrelation
ExistingPsychological Fep Associatingapsychologicalfunctionwithanexistingelementorspaceoraspatialrelation
ConsiderationPsychological Fcp Thinkingofanpsychologicalfunctiontobeimplementedindependentlyofvisualfeatures
Conceptual C Cognitiveactionswhicharenotdirectlycausedavisuospatialfeatures
Coevolution Ce Preferential(likedislike)oraesthetical(beautifulugly)assessmentofthePactionsorFactions
SetupGoalactivities Cg
Abstracted issues out of particular situations in design representation which are general
enoughtobeacceptedviathedesignprocessthoroughlyasamajordesignnecessity
GoalBrief Cgb Other Goalsbasedontherequirementsofthedesignbrief
GoalExplicit Cge Sinv Goalsintroducebytheexplicitknowledgeorpreviouscases
GoalPast Cgp Sinv Comingoutthroughpastgoals
GoalTacit Cgt Sinv Thegoalsthatarenotsupportedbyexplicitknowledge,givenrequirements,orpreviousgoals
GoalConflict Chc Sinv Goalsdevisedtoovercomeproblemswhicharecausedbypreviousgoals
GoalReapply Cgr Other Goalstoapplyalreadyintroducedfunctionsinthenewsituation
GoalRepeateD Cgd Other
Goalsrepeatedthroughsegments
3.2.3 Measurement of design protocols and testing hypotheses

After performing segmentation process and developing the coding scheme, based on recorded videos and transcribed
media we assigned related codes to every segment. Protocol analysis and interpretation starts only after assigning
related codes to every segment. In this study we relied on both descriptive and inferential statistics to analyze and
interpret the collected data. Graphs and charts have been employed in descriptive statistics to explore the meaningful
44
9
th

patterns of changes in designers cognitive actions. On the other hand, the inferential statistics has been employed
for testing the assumed hypotheses. For comparison purpose we used Wilcoxon ranks test which is equivalent to
paired sample t-test for non-parametrically distributed data. Then, for the purpose of testing relationship Chi-square
test have been selected for the cases in which the independent variables are categorical (nominal) and the dependent
variables are non-parametrically distributed ratio scaled values. For interpretation of the results, we compared the
test statistics against the significant value which is at (.05) for social sciences.
4. RESULTS AND ANALYSIS OF COLLECTED EMPIRICAL PROTOCOL DATA
This section presents the results and analysis data collected during the empirical study which was explained in
Section 3. The analysis in this chapter mostly hinges around four different levels of designers spatial cognition:
physical-action (P), perceptual-actions (Pe), functional-actions, and conceptual-actions (FC).
4.1 Overview of the coded data
Table 2 shows the mean value and standard deviation of segment duration for six sessions by three designer groups
as well as average value of each design method. For maintaining homogeneity of the results we tried to balance the
total time which every group spent for each of the sessions. The conducted Wilcoxon ranks test on coded data show
significant reduction in the average length of designers utterances during 3D sketching process compared to those
in traditional sketching process. According to Kim and Maher (2008) this could be considered as a good
phenomenon since it has potential for decreasing the load of designers mental cognitive processes.
TABLE. 2: Duration of segments for both traditional (Man) and 3D sketching (Digi) sessions
Session Total time (s) Segment num. Mean (s)/Std. Z value/Sig.
Pairs Ave. (Man) 10148 1941 14.61/20.67
Pairs Ave. (Digi) 10422 2581 10.78/14.80
8.77
(a)***
/.000
4.2 Analysis of designers spatial cognition
Table 3 exposes the occurrence frequency percentage of Physical-actions (P-actions), Perceptual-actions (Pe-
actions) and Functional-Conceptual actions during both manual and 3D sketching design sessions. The analysis and
of occurrence percentage of all three mentioned action categories are presented in related following sections.
TABLE. 3: Occurrence frequency percentage of cognitive activities (CA) for Manual and 3D Digital design sessions
Cognitive Activities

P-actions
Pe-actions
FC-actions
Total
Manual % within the whole CA 74.9% 11.6% 13.5% 100.0%
Mode CA
Digital % within the whole CA 69.1% 11.8% 19.1% 100.0%
4.2.1 Physical-actions
So far, based on the results coming from the selected sample we have one confirmed hypothesis which infers that
the proposed 3D sketching methodology can increase the total amount of the designers external cognitive activities
compared to the traditional design tools. For interpretation of this finding, the study needs to test the percentage of
the total P-actions among the whole cognitive actions. Table 3 shows decrement for percentage of the P-actions in
3D sketching sessions (69.1%) in comparison to those in traditional design sessions (74.9%). Conducted Chi-square
test confirms the assumed significance (X
2
=12.851, df=1, r<.001). As a consequence, the second hypotheses could
be confirmed. Based on results from the selected sample it can be inferred that compared to the traditional design
tools, the proposed 3D sketching methodology can decrease the occurrence frequency percentage of P-actions
among the entire designers external cognitive activities.
4.2.2 Perceptual-actions
In our coding system two major concerns are related to Pe-actions. The first concern is with the absolute number and
percentage of occurrence frequency of whole Pe-actions. The other concern is related to the occurrence of the
unexpected discoveries codes compared to the occurrence of the other perceptual codes. Figure 4 illustrates the
occurrence frequency percentage of the designers unexpected discoveries and the other Pe-actions during both
sessions. Results show that in all 3D sketching sessions there is an obvious increment (X
2
=9.889, df=1, r<.01) in
45
9
th

percentage of unexpected discoveries compared to the other Pe-actions. Moreover, according to the conducted
Wilcoxon ranks test to compare total amount of perceptual activities within similar allotted time, when using 3D
sketching methodology designers performed significantly more Pe-actions (Z=-6.489, p<.001) compared to what
they did during the traditional design sessions. Finally, in terms of visual analysis of the processes, referring to the
occurrence frequency scatter bars of Pe-actions reveals that in the two of the three cases occurrence of Pe-actions are
more consistent throughout the 3D sketching sessions compared to those in traditional sessions.

FIG. 4: Occurrence frequency percentage of the designers unexpected discoveries (Unex. Disc.), and the other Pe-
actions

4.2.3 FC-actions
In this section designers cognitive activities are categorized into three major categories. The first part comprises
designers all evaluations of their previous physical, perceptual, and functional actions. The second category belongs
to designers all functional and set-up goal activities. In this section, regardless of whether those actions are about
assigned functions or their set-up goals, all the codes are analyzed under the following two groups: 1) situative-
inventions, and 2) other FC-actions. Figure 5 illustrates the occurrence frequency percentage of all three groups of
the codes for all six traditional and 3D sketching design sessions. From the bar charts it can be concluded that
although the occurrence tendency of situative-inventions codes is almost the same (X
2
=1.509, df=1, r>.05) for all the
traditional and 3D sketching sessions, there is a huge increment for occurrence tendency of co-evolutions codes
during 3D sketching sessions compared to those in traditional session (X
2
=53.555, df=1, r<.001). This increment can
be considered as a consequent of the more explicit representations during 3D sketching sessions. Moreover, the
conducted chi-square test shows that there is a significant increment for occurrence percentage of FC-actions among
total cognitive activities (X
2
=17.179, df=1, r<.001). Finally, in analysing the processes visually, the occurrence
frequency scatter bars of FC-actions reveal that occurrence of FC-actions is more consistent throughout the 3D
sketching sessions compared to those in traditional sessions.

FIG. 5: Occurrence frequency percentage of the designers co-evolutions (Co-evol.), situative-inventions (S-inv),
and the other functional-conceptual (FC) actions during all six traditional (M) and 3D sketching (D) sessions
5. CONCLUSIONS
The main aim of this experiment was to provide objective and empirical evidence for the proposed VR-based 3D
sketching interface improves the designers spatial cognition during conceptual architectural design phase. In this
46
9
th

experiment the focus was on designers cognitive actions and the hypotheses were being tested relying on the
designers actions. The codes assigned to the different segments were considered as our units of analysis. Although
this experiment was made up of three pairs of designers performing six design sessions in total, the experiment
provides adequate data for observing overall designingly trends and actions. Besides, we were guided by Clayton et
al.s (1998) recommendations in validating our results. Moreover, during our exploratory study we had observed
consistent improvements in the main five aspects of design sessions and spatial cognition across the three pairs that
further validated the claim that 3D sketching interface facilitates better quality of designing.
The study found that in 3D sketching sessions, the increased integration of the physical actions with mental
perceptions and conceptions would lead to occurrence of epistemic actions which improves the designers spatial
cognition. The results support (Kirsh and Maglio 1994) argument that the epistemic actions facilitated by the rich
interface would offload the designers mental cognition partly into the physical world, hence allowing them freer
mind to create more design ideas. Moreover, the 3D sketching interface improves the designers perception of visuo-
spatial features, particularly in terms of unexpectedly discoveries of spatial features and relationships. The
phenomenon we observed is explained by Schn (1983a) whereby there exist an association between mental
cognition and the perception of physical attributes that could stimulate creativity and offload the mental load.
Furthermore, the results support from Suwa et al.s (2000) arguments to explain how unexpected discoveries can
lead to more creativity and also to the occurrence of more situative inventions.
In terms of functional-conceptual actions of the design process, we posit that 3D sketching interface would improve
the designers problem finding behaviours as well as improving their co-evolutionary conceptions of their
perceptions and problem findings. Suwa et al.s (2000) explain these behaviours as situative-inventions and argue
how the increased percentage of the co-evolutionary and situative-inventions actions can lead towards improved
creativity in 3D sketching design session. In conclusion, we argue that the emerging VR technologies are capable to
facilitate physical senses beyond the visual aspects of the design artefact by offering a new generation of promising
CAD tools which are constantly in touch with designers cognition during conceptual architectural design process.
6. ACKNOWLEDGMENTS
We acknowledge that this research is a part of doctoral study by the first author at Universiti Putra Malaysia (UPM)
which is partly sponsored by UPM's Graduate Research Fellowship (GRF). We also would like to gratefully
acknowledge contributions of the fifth year architectural students in the Semester 2 2008/2009 at the Faculty of
Design and Architecture, UPM. We also acknowledge the contributions of Prof. Dr. Mohd Saleh B. Hj Jaafar,
Associate Prof. Dr. Rahmita Wirza Binti O. K. Rahmat, and Dr. Muhamad Taufik B Abdullah during this study.
7. REFERENCES
Basque Research. (2007). "Using Computerized Sense Of Touch Over Long Distances: Haptics For Industrial
Applications." ScienceDaily.
Bilda, Z., and Demirkan, H. (2003). "An insight on designers' sketching activities in traditional versus digital
media." Design Studies, 24(1), 27-50.
Brewster, S. (2001). "The Impact of Haptic Touching Technology on Cultural Applications." Glasgow Interactive
Systems Group, Department of Computing Science, University of Glasgow, Glasgow.
Craft, B., and Cairns, P. "Work interaction design: Designing for human work." IFIP TC 13.6 WG conference:
Designing for human work, Madeira, 103-122.
Cross, N. (2007). Designerly Ways of Knowing (paperback edition), Birkhuser, Basel, Switzerland.
Cross, N., Christiaans, H., and Dorst, K. (1996). Analysing Design Activity, NY: Wiley & Sons, New York.
Cross, N., and Dorst, K. (1999). "Co-evolution of problem and solution space in creative design " Computational
models of creative design, J. S. Gero and M. L. Maher, eds., Key Centre of Design Computing, University of
Sydney, Sydney, 243-262.
Ericsson, K. A., and Simon, H. A. (1993). Protocol analysis: verbal reports as data, MIT Press, Cambridge
Fitzmaurice, G. W. (1996). "Graspable user interfaces," University of Toronto Toronto.
47
9
th

Gero, J. S., and Damski, J. (1997). "A symbolic model for shape emergence." Environment and Planning B:
Planning and Design, 24, 509-526.
Goldschmidt, G., and Porter, W. L. (2004). Design representation, Springer, New York.
Ibrahim, R., and Pour Rahimian, F. (In review). "Empirical comparison on design synthesis strategies by
architectural students when using traditional cad and manual sketching tools." Automation in Construction,
Special Issue for CONVR08's Selected Papers.
Kan, W. T. (2008). "Quantitative Methods for Studying Design Protocols," The University of Sydney, Sydney.
Kim, M. J., and Maher, M. L. (2008). "The impact of tangible user interfaces on spatial cognition during
collaborative design." Design Studies, 29(3), 222-253.
Kirsh, D., and Maglio, P. (1994). "On distinguishing epistemic from pragmatic action." Cognitive Science, 18(4),
513-549.
Kwon, J., Choi, H., Lee, J., and Chai, Y. " Free-Hand Stroke Based NURBS Surface for Sketching and Deforming
3D Contents." PCM 2005, Part I, LNCS 3767, 315 326.
Lloyd, P., Lawson, B., and Scott, P. (1995). "Can concurrent verbalization reveal design cognition?" Design Studies,
16(2), 237-259.
Regenbrecht, H., and Donath, D. (1996). "Architectural Education and VRAD." uni-weimar.
Schn, D. (1983a). The Reflective Practitioner: How Professionals Think in Action, Temple Smith, London.
Schn, D. (1992). "Designing as reflective conversation with the materials of a design situation " Knowledge-Based
Systems, 5(1), 3-14.
Stricker, D., Klinker, G., and Reiners, D. (2001). "Augmented reality for exterior construction applications."
Augmented reality and wearable computers, W. Barfield and T. Caudell, eds., Lawrence Erlbaum Press, 53.
Suwa, M., Gero, J. S., and Purcell, A. T. (2000). "Unexpected discoveries and S-inventions of design requirements:
important vehicles for a design process " Design Studies, 21(6), 539-567.
Suwa, M., and Tversky, B. (2001). "How do designers shift their focus of attention in their own sketches? ."
Diagrammatic reasoning and representation, M. Anderson, B. Meyer, and P. Olivier, eds., Springer, Berlin,
241-260.
Tversky, B. (2005). "Functional significance of visuospatial representations." Handbook of higher-level visuospatial
thinking, P. Shah and A. Miyake, eds., Cambridge University Press, Cambridge, 1-34.
van Someren, M. W., Barnard, Y. F., and Sandberg, J. A. C. (1994). The Think Aloud Method: A Practical Guide to
Modelling Cognitive Processes, Academic Press, London.
Visser, W. (2004). "Dynamic aspects of design cognition: elements for a cognitive model of design." Theme 3A-
Databases, Knowledge Bases and Cognitive Systems, Projet EIFFEL, France.

48
9
th

ANALYSIS OF DISPLAY LUMINANCE FOR OUTDOOR AND MULTI-
USER USE
Tomohiro Fukuda, Associate Professor,
Graduate School of Engineering, Osaka University;
fukuda@see.eng.osaka-u.ac.jp and http://y-f-lab.jp/fukudablog/
ABSTRACT: The use of digital tools outdoors is anticipated, but a problem exists that when an ordinary PC display
is used it is hard to see because of outside light. To clarify the cause, three elements of the display were evaluated,
namely luminance, the contrast ratio, and the viewing angle. Five displays were assessed by using a luminance
meter, and the three elements of each display were measured in a darkroom. To decrease the various factors
affecting luminance measurement outdoors, the illuminance change outdoors, the influence of sunlight, and the
influence of the ambient surroundings were considered. Also, using experimental methodology that reduced these
factors, data about the three elements were acquired outdoors. Future work will need to clarify the element of the
luminance of the display outdoors.
KEYWORDS: Display for outdoor, luminance, contrast ratio, viewing angle, digital tool.
1. INTRODUCTION
Digital tools for multi-media purposes including MRMixed Realityare expected to be used outdoors by many
users. However, in general, the medium of paper is used in design studies and by tour guides due to the various
problems that digital tools have (left and middle of FIG.1). In this study, from the various problems affecting digital
tools, the problem of the display is targeted. In general, the display is not easy to see outdoors due to the influence of
the outside light (Right of FIG.1).

FIG. 1: Sharing information outdoors using paper (left and middle); State of display outdoors (right).
Digital tools used outside by individuals, such as cellular phones, PDAs or HMDs (Head Mounted Display), were
not the target of this study. Instead the focus was on digital tools with which a number of people can share
information. Many papers have reported on digital tools used outdoors (Feiner, 1997; Behringer 2000; Julier 2000;
Baillot 2001; Kuo, 2004; Onohara, 2005). However, those papers described the feature development of the digital
tools, and did not consider the problem of ease of viewing the display outdoors.
The author has developed an MR system which includes a video image displaying a present image, and a real-time
3DCG image displaying images of objects or scenes that do not exist, such as design proposals or demolished
buildings in real time (Fukuda, 2006; Kaga, 2007). The set-up of this system for outdoor, multi-user, and mobile use
includes a tablet PC, MR software, a live camera, RTK-GPS (Real Time Kinematic - Global Positioning System),
and a 3D motion sensor. This system is expected to be used for city tours or design studies in the areas of education,
research, and practice.
In the presented paper (Fukuda, 2009), a problem of the developed MR system was shown, namely that the display
is hard to see outdoors because of the influence of the outside light. To grasp the problem quantitatively, three
displays were measured. However, this was a preliminary study and further research on the conditions of display
experiment used outdoors is needed.
49
9
th

This study aims to examine the problem of the display being hard to see outdoors. Two displays in addition to those
described in the presented paper (Fukuda, 2009) were added to improve the research method. In addition, the
experiment was carried out on five displays after the problem and measurements of the experiment on outdoor were
clarified.
2. DISPLAY EVALUATION ELEMENT
The brightness of a display is expressed with a luminance and a contrast ratio. Luminance is a photometric measure
of the luminous intensity per unit area of light travelling in a given direction. Where F is the luminous flux or
luminous power (lm), is the angle between the surface normal and the specified direction, A is the area of the
surface (m
2
), and is the solid angle (sr), Luminance L (cd/m
2
) is defined by
(1)
The contrast ratio is the luminosity ratio of the maximum luminosity L
max
{(R, G, B) = (255, 255, 255)} and the
minimum luminosity L
min
{(R, G, B) = (0, 0, 0)} on a display. The higher the contrast ratio, the greater the difference
between L
max
and L
min
is. To raise the contrast ratio, L
max
is enlarged or L
min
is unlimitedly brought close to 0.
Contrast ratio CR is defined by
(2)
In addition, a viewing angle that assumes about ten people can see the display is set. The viewing angle is the
luminosity ratio of the display front luminosity L
f
and the diagonal 45 display degree luminosity L
s
. The lower the
value, the smaller the difference between the luminance from the front and the diagonal luminance. That is, the
screen can be seen very well from a diagonal viewpoint. Viewing angle VA is defined by
(3)
3. EXPERIMENT AND RESULT
3.1 Whole image of experiment
To grasp the characteristics of the display for outdoor use, luminance L was measured in the darkroom and outdoors.
The contrast ratio CR and the viewing angle VA were calculated based on the measured luminosity value. Display
makers usually provide the dark room contrast ratio, which is a contrast ratio usually measured in a dark room where
illuminance is 0. The dark room contrast ratio is the standard numerical value. However, since it is significantly
influenced by outdoor daylight, just the dark room contrast ratio of evaluation of an outdoor display is inadequate.
Therefore, it is necessary to measure the ambient contrast ratio, which is a contrast ratio that adds and measures the
conditions of fixed outdoor daylight. The luminance meter used was an LS-100 by Konica Minolta Sensing, inc. and
the illuminance meter used was a T-10 by Konica Minolta Sensing, inc.
The procedure used in the experiment is shown below:
The display and the luminance meter were set up in the darkroom as shown in FIG. 2.
The power supply on the display was switched off.
Luminance at the center of the screen was measured five times.
The power supply on the display was switched on.
A black screen {(R,G,B)=(0,0,0)}, which has the lowest luminance, was displayed, and the luminance
at the center of the screen was measured five times.
50
9
th

A white screen {(R,G,B)=(255,255,255)}, which has the highest luminance, was displayed, and
luminance at the center of the screen was measured five times.
The positions in which luminance was measured were the front and a diagonal 45-degree position
relative to the display.
The specifications of the display are shown in TABLE 1. The measurement conditions are shown in TABLE 2. FIG.
2 shows the plans of the experiment and a photo.
TABLE.1:Specifications of display.
Display1
NEC VersaPro
VY11F/GL-R
Display2
Lenovo X41Tablet
Display3
Sony VAIO
VGN-SZ94PS
Display4
NEC Sheild PRO
FC-N22A
Display5
Sony XEL-1
Picture

CPU Intel Pentium M
1.1GHz
Intel Pentium M
1.6GHz
Intel Core2 Duo
T7800 2.6GHz
Intel ULV U7500
1.06GHz
RAM 512MB 1.49GB 2GB 2GB
Graphic Memory
(VRAM)
ATI MOBILITY
RADEON 7500
(32MB)
Intel 915GM Express
(96 MB)
NVIDIA GeForce
8400M GS
(256MB)
Intel GMAX3100

Weight (kg) 0.855 1.88 1.75 2.5 2.0
Display size (inch) 10.4 12.1 13.3 12.1 11
Time to market Aug., 2004 Jul., 2005 Dec., 2007 Aug., 2008 Oct., 2008
TABLE. 2: Measurement conditions.
Dark-room experiment Outdoor pilot experiment Outdoor experiment
Illuminance on the ground (lx) 0 22,030 107,800 29,470 83,300
Measurement time Jul. 4, 2009, at 20:00 22:00 Jul. 3 and 6, 2009, at 8:00 11:30 Jul. 9, 2009, at 9:45 12:00
Weather Fine weather Obscured sky Obscured sky
Latitude, Longitude 34.822623, 135.522781 (Osaka, Japan)

FIG. 2: Plan of dark-room experiment (upper left) and outdoor experiment (upper middle) (o: display power-off; k:
black image; w: white image); Photo of outdoor experiment (right).
51
9
th

3.2 Darkroom experiment
3.2.1 Experiment method
The method of the experiment outdoors was the same as described in Chapter 3.1. It was confirmed that the
illuminance of the darkroom was 0 lx by using the illuminance meter.
3.2.2 Result
The measurement was carried out on each pattern five times. The mean value of the value obtained for the five
measurements was used for the analysis. FIG.3 shows the luminance comparison on each display. FIG.4 shows the
contrast ratio comparison on each display. FIG.5 shows the viewing angle comparison on each display.
The value of the screen luminance when power was off was the lowest in each display, as shown in FIG.3. Each
value was close to 0. When the screen was black, L
d5
indicated the same value as the luminance when power was
off. This shows that jet-black can be expressed with the display5 made by the organic EL display. The luminance
value from the display1 to the display4 was higher than that of the display5 at the black screen. This is because it is
difficult to completely remove the influence of the backlight from the screen since it forms a basic part of the
principle of operation of the liquid crystal panel. When the screen is white, the L
d1
used as a display of the MR
system indicated 77.854cd/m
2,
, which was the lowest value. L
d4
indicated 394.48cd/m
2
, which was the highest.
The contrast ratio CR
d1
-CR
d4
of the liquid crystal display (Display1-4) was 346.9-577.4, as shown in FIG.4. On the
other hand, CR
d5
of the organic EL display (Display5) was 147857.1, which was a considerably high value.
The viewing angle of the display was 2.21-5.33, as shown in FIG.5. The value of VA
d5
was the lowest with 2.21.

FIG. 3: Dark-room luminance (L
d
) comparison on each display.

FIG. 4: Dark-room contrast ratio (CR
d
52
9
th

FIG. 5: Dark-room viewing angle (VA
d
3.3 Outdoor pilot experiment
Outdoors, there are many parameters that make it difficult to obtain a constant result. Such experimental problems
were clarified by conducting a pilot study using display2 and display3.
3.3.1 Illuminance change outdoors
The illuminance of the ground was measured in pilot experiment 1. In regard to the experiment time, the illuminance
was measured for 2 hours and 30 minutes. This was done on July 3 and July 6, 2009. The time was from 9:00 to
11:30. The weather was overcast. The result is shown in FIG. 6.
FIG. 6 clearly shows that illuminance outdoors is not constant. On July 3, the illuminance meter indicated 35,100-
107,800lx. On July 6, the illuminance meter indicated 22,030-72,100lx. The illuminance changes outdoors influence
the luminance of the display. It is thought this change is caused by changes in the weather and the difference in the
time of the experiment. It is possible to select dates when the weather is steady, and to shorten the time of the
experiment.

FIG. 6: Illuminance change in outdoor.
3.3.2 Influence of sunlight
The influence of sunlight was examined in pilot experiment 2. To reduce the influence on the luminance of the
display, the display showed a black screen, and a black board was set up in the background. Luminance was
measured from the front, from a diagonal 45-degree position, and from a diagonal 135-degree position relative to the
display. The result is shown in FIG. 7.
53
9
th

FIG. 7 clearly shows that the luminance of the display measured from the diagonal positions was higher than that
measured from the front. This is thought to be due to specular reflection when measuring diagonally because of the
relation to the angle of incidence of sunlight. The luminance of the sun is about 210
9
cd/m
2
, and it is overwhelming
compared with the brightness of the display. The display was set up with the sun behind it so that the influence of
direct sun could be reduced.

FIG. 7: Influence of sunlight (-fk: measurement from the front; -s1k: measurement from the diagonal 45 degrees; -
s2k: measurement from the diagonal 135 degrees).
3.3.3 Influence of ambient surroundings
The influence of the ambient surroundings was examined in pilot experiment 3. The display was set up with the sun
behind it, taking account of the result of pilot experiment 2. Then, the display was changed to a black screen, and
measured from the front. Different ambient surroundings that were reflected on the display were used and measured,
namely an unchanged state, a white board, and a black board. The result is shown in FIG. 8.
FIG. 8. clearly shows that the luminance of the display was the highest with the white board, and lowest with the
black board. That is, the ambient surroundings influence the luminance of the display. It is possible to stabilize the
scenery that is reflected on the display. The black board, which caused the least reflections, was set as the ambient
surroundings.

FIG. 8: Influence of ambient surrounding (-fn: a state without change; -fw: a state with a white board; -fk: a state
with a black board).
As a result of these considerations, the following experimental conditions were applied:
The change in the illuminance was noted.
54
9
th

The display was set up with the sun behind it.
The black board was used for the background reflected on the display.
3.4 Outdoor experiment
3.4.1 Experiment method
The method of the outdoors experiment was the same as that described in Chapter 3.1. The illuminance of the
ground was measured at the same time as measuring the luminance.
3.4.2 Result
FIG.9 shows the luminance of each display and the illuminance of the ground. When the display was set to a black
screen, the luminance was 224.3-239cd/m
2
in display1, 75.96-81.6cd/m
2
in display2, 99.98-103.9cd/m
2
in display3,
229.9-241.5cd/m
2
in display4, and 243-258.3 cd/m
2
in display5. All the values were higher than that of the
darkroom. It is thought that the outside light influenced this. When the display was set to a white screen, the
luminance was 320.2-337.3 cd/m
2
in display1, 187.9-199.5 cd/m
2
in display2, 229.4-231.5 cd/m
2
in display3, 677.4-
726.5 cd/m
2
in display4, and 326.4-333.6 cd/m
2
in display5. All the values were higher than that of the darkroom. It
is thought that the outside light influenced this.
The ambient contrast ratio CR
bn
of all displays was 1.314-3.010 from FIG.10. The viewing angle VA
bn
of all displays
was 0.822-1.919 as shown in FIG.11.
55
9
th

FIG. 9: Outdoor luminance (L
b
) and the illuminance comparison on each display (-fo: display power-off; -fk: black
screen; -fw: white screen).

FIG. 10: Outdoor contrast ratio (CR
o
56
9
th

FIG. 11: Outdoor viewing angle (VA
o
4. DISCUSSION
Outdoors, there are many factors that make it difficult to obtain a constant result. In this study, to decrease the
variation factor, the illuminance change outdoors, the influence of sunlight, and the influence of the ambient
surroundings were considered, as described in Chapter 3.3. Also, experimental methodology that reduced these
variation factors as much as possible was used in Chapter 3.4.
When FIG.9 is compared with FIG.3, the luminance L of the displays differs greatly in the dark room and outdoors
in the sunlight. The luminance of a white screen is higher than that of a black screen in each display though it is
necessary to remember that the illuminance can be uneven. Moreover, the luminance of a switched off screen is
higher than that of a black screen, excluding display5. The luminance of a white screen is higher than that of the
darkroom. This reason is that the influence of the outside light is received. Whether the ratio of the luminance of the
outdoor display is an influence of the luminescence of the liquid crystal display or of the outside light cannot be
judged from the measuring method of the luminance used in this study. When FIG.10 is compared with FIG.4, the
contrast ratio CR also differs greatly in the dark room and outdoors in the sunlight. One of the causes of this is that
the luminance of a black screen is high compared to the influence of the outside light. When FIG.11 is compared
with FIG.5, the viewing angle VA also differs greatly in the dark room and outdoors in the sunlight. As Chapter 3.1
described, when evaluating the display used outdoors, measurement of the ambient contrast ratio is important.
This experiment was executed in a certain specific time. That is, since outdoor illuminance constantly changes, it is
difficult to reproduce the result of the experiment. Therefore, it is necessary to build a display evaluation system
which can reproduce and measure outdoor daylight conditions.
5. CONCLUSION AND FUTURE WORK
The results achieved in the present study are as follows:
The outdoor use of digital tools such as MR is anticipated, and there is a problem that the screen is
hard to see when a normal display is used. To clarify the reason for this, luminance, the contrast ratio,
and the viewing angle of the display were evaluated.
Five displays were measured using a luminance meter, and as part of the luminance data, the contrast
ratio, and the viewing angle of each display were acquired in the darkroom.
Outdoors, there are many factors that make it difficult to obtaining a constant result. To decrease the
variation factors, the illuminance change outdoors, the influence of sunlight, and the influence of the
ambient surroundings were considered. Also, using experimental methodology that reduced these
variation factors as much as possible, data on luminance, the contrast ratio, and the viewing angle of
each display were acquired outdoors.
Future works could investigate the following areas:
Whether the ratio of the luminance of the outdoor display is an influence of the luminescence of the
liquid crystal display or of the outside light cannot be judged from the luminance measuring method
57
9
th

used in this study. A clear relation was not obtained although clarification was attempted in the
experiment by measuring luminance in the switched off state. It is necessary to examine the character
of the material that composes the display.
To understand the outdoor characteristics of each display, the construction of an evaluation system
that artificially produces the outdoor environment is thought necessary.
6. ACKNOWLEDGEMENT
The author would like to thank Mr. Wei Cheng Lin and Mr. Tian Zhang who are research students of Osaka
University for supporting these experiments.
7. REFERENCES
Behringer, R. et al. (2000). "A Wearable Augmented Reality Test-bed for Navigation and Control, Built Solely with
Commercial-off-the-Shelf (COTS) Hardware.", Proc. 2nd Intl Symp. Augmented Reality 2000 (ISAR 00),
IEEE CS Press, Los Alamitos, Galif., 12-19.
Baillot, Y., Brown, D., and Julier, S. (2001). "Authoring of Physical Models Using Mobile Computers.", Proc. Intl
Symp. Wearable Computers, IEEE CS Press, Los Alamitos, Galif.
Feiner, S., MacIntyre, B., Hllerer, T., Webster, A. (1997). "A touring machine: Prototyping 3D mobile augmented
reality systems for exploring the urban environment.", Personal and Ubiquitous Computing, Volume 1,
Number 4, Springer London, 208-217.
Fukuda, T., Kawaguchi, M., Yeo, W.H., and A. Kaga, A. (2006). "Development of the Environmental Design Tool
"Tablet MR" on-site by Mobile Mixed Reality Technology.", Proceedings of The 24th eCAADe (Education
and Research in Computer Aided Architectural Design in Europe), 84-87.
Fukuda, T. (2009). "Analysis of a Mixed Reality Display for Outdoor and Multi-user Implementation ", 4th
ASCAAD Conference (Arab Society for Computer Aided Architectural Design), 323-334.
Julier, S. er al. (2000). "Information Filtering for Mobile Augmented Reality.", Proc. Intl Symp. Augmented Reality
2000 (ISAR 00), IEEE CS Press, Los Alamitos, Calif., 3-11.
Kaga, A., Kawaguchi, M., Fukuda, T., Yeo, W.H. (2007). "Simulation of an Historic Building Using a Tablet MR
System.", Proceedings of the 12th International Conference on Computer Aided Architectural Design
Futures, Sydney (Australia), 45-58.
Kuo, C.G., Lin, H.C., Shen, Y.T., Jeng, T.S. (2004). "Mobile Augmented Reality for Spatial Information
Exploration.", Proceedings of The 9th International Conference on Computer Aided Architectural Design
Research in Asia (CAADRIA2004), 891-900.
Onohara, Y., Kishimoto, T., (2005). "VR System by the Combination of HMD and Gyro Sensor for Streetscape
Evaluation.", Proceedings of The 10th International Conference on Computer Aided Architectural Design
Research in Asia (CAADRIA2005), vol. 2, 123-128.
58
9
th

A PROPOSED APPROACH TO ANALYZING THE ADOPTION AND
IMPLEMENTATION OF VIRTUAL REALITY TECHNOLOGIES FOR
MODULAR CONSTRUCTION
Yasir Kadhim, Graduate Research Student
Department of Civil Engineering, University of New Brunswick, Canada;
y26f6@unb.ca
Jeff Rankin, Associate Professor
and M. Patrick Gillin Chair in Construction Engineering and Management
Department of Civil Engineering, University of New Brunswick, Canada;
rankin@unb.ca
Joseph Neelamkavil, Senior Research Officer
National Research Council of Canada, Centre for Computer-assisted Construction Technologies
Joseph.Neelamkavil@nrc-cnrc.gc.ca
Irina Kondratova, Group Leader
National Research Council of Canada, Institute for Information Technology
Irina.Kondratova@nrc-cnrc.gc.ca
ABSTRACT: To achieve successful adoption and implementation of process technologies in the construction
industry requires a better understanding of practices of innovation management. Defining innovation as the process
of applying something new, a research project is being undertaken to contribute to a better understanding of its
concomitant practices. The project focuses on virtual reality (VR) technologies within a specific application of
modular construction. From a potential adopters perspective, the process of technology adoption and
implementation is often less than satisfactory. The research project is addressing this by furthering the
understanding of the innovation process in a series of case studies. This paper presents work in progress to this end
by providing the background on assessing management practices, the link between VR technologies and modular
construction and early results of case study activities. To date, providing the functionality to enhance
communication has been identified as the best fit for the case studies of applying virtual reality technologies to the
process of modular construction engineering and management. The conceptual framework of assessing innovation
management practices that employs the concept of capability maturity is presented as a predictive indicator for the
adoption and implementation that is to follow.
KEYWORDS: Virtual reality, modular construction, innovation management, technology adoption
1. INTRODUCTION
Many practitioners and researchers alike agree that the architectural engineering and construction (AEC) industry
can improve its overall performance (measured in terms of cost, time, safety, quality, sustainability, etc.) by creating
a better business environment that encourages innovation. Innovation is defined in this context as application of
technology that is new to an organization and that significantly improves the design and construction of a living
space by decreasing installed cost, increasing installed performance, and/or improving the business process (e.g.,
reduces lead time or increases flexibility) (Tooles 1998).
The research described focuses on process technologies within the AEC industry as a class of innovations. Process
technologies are loosely defined as any tool or technique that supports the management of a civil engineering project
during execution from concept, through design, construction and operation, to decommissioning. This focus area
presents some interesting challenges and some corresponding gaps in the knowledge area. From a potential
adopters perspective, it is difficult to objectively assess process technologies for adoption and implementation as
there are not many decision-making tools and techniques for industry to properly identify needs and match
corresponding solutions. Overcoming this challenge requires a direct link to performance, whether at the
organization, project or industry level, whereas currently, the focus has been on operational savings. For example, it
59
9
th

is easy to measure the time savings realized by recording information electronically versus on paper; however, an
assessment of the knock-on positive effects on performance, by having this information conveniently archived, is a
bit more difficult to measure. Some of the questions to answer include: how do we improve the performance of the
AEC industry through the effective development and appropriate adoption and implementation of process
technologies; what are the techniques to support practitioners in the analysis of new process technologies; and what
contributes to a strategy for increasing the impact and rate of process technology adoption within the AEC industry?
The approach that has been taken is to assess the performance of the process of construction while taking into
account the management practices being applied. A modest research project is being conducted jointly by the
University of New Brunswicks Construction Engineering and Management Group (UNB CEM) and the National
Research Council of Canadas Centre for Computer-assisted Construction Technologies (NRC-CCCT). The short
term research objectives are to study the implementation of a specific advanced process technology (i.e., virtual
reality technologies) for a specific scenario in the industry (i.e., modular construction). The research is also intended
to contribute to a broader research program of more formally assessing the impact of innovation management
practices on industry performance. The research project hypothesis states that the maturity of management practices
at various levels within an organization, with respect to process technologies, can be measured and correlated with
the performance in the adoption of technologies.
The research project contains the following objectives: 1) defining the basic competency requirements beyond the
standard construction engineering and management domain, for both modular construction and virtual reality
technologies;, 2) determining the basic challenges in the adoption and implementation of modular construction and
virtual reality technologies;, 3) establishing the details of a case study through the definition of usage scenarios for
the application of virtual reality technologies to modular construction;, 4) developing the technological environment
required for the usage scenarios through the configuration of existing technologies, and 5) capturing the case study
and assessing the use of virtual reality technologies to further a definition of innovation management.
The paper reports on work in progress by first providing background on the approach to assessing performance and
practices, topics of virtual reality (VR) technologies and their advantages and modular construction approaches and
their advantages and challenges. The details of the industry case studies being used in the research project are
provided, followed by a description of the methodology being used to assess the adoption and implementation of the
technology contributing to the assessment of innovation management practices at an organizational level.
2. ASSESSING PERFORMANCE AND PRACTICES
To place this research in the context of assessing performance in the construction industry, Figure 1 depicts a high
level process view of construction (Fayek et al. 2008). Measuring the performance of the process at some level of
granularity (e.g., activity, project, organizational, sector, industry) typically measures the ratio of outputs to inputs
(A to A) and the extent to which objectives are achieved (C), under a given set of conditions (B), while employing a
set of practices (D). The research described in this paper explores innovation management practices (D) and it does
so at the organizational level of granularity. The aggregation (e.g., to a sector level) and/or specialization (e.g., to an
activity level) of the assessment is not covered in the scope of the framework developed. In order to study this, a
specific innovation (VR technologies) and scenario for application (modular construction) is required.
Process maturity modelling gained its greatest attention in the software manufacturing industry (Finnemore et al.
2000) and is based on the earlier concepts of process improvement such as the Shewhart plandocheckact cycle,
as well as on Philip Crosbys quality management maturity grid which describes five evolutionary stages in
adopting quality practices (Crosby 1979). Researchers at Carnegie Mellon University used this concept in the
development of the Capability Maturity Model (CMM) (Paulk et al. 1995). CMM highlights the five thresholds of
maturity through which a process must transition in order to be sustainably improved. Initially, a process is (1)
chaotic or ad-hoc and must be made (2) repeatable, after which it must be (3) defined or standardized. The process
must then be (4) managed, i.e., measured and controlled. Ultimately, the process must be (5) optimized, i.e., it must
be continuously improved via feedback and through the use of innovative ideas and technologies. The assessment of
the maturity of a process at the organizational level entails determining the extent to which the process is defined,
managed, measured and controlled; and this is commonly achieved by observing the practices within the
organization. A more general definition is that maturity may be viewed as a combination of actions, attitudes, and
knowledge rather than constraining the definition to a single set of actions or procedural norms (Andersen and
60
9
th

Jessen 2003). Closer to the construction industry and management of projects are more recent maturity models that
include the Project Management Process Maturity (PM)2 Model (Kwak and Ibbs 2002), the Standardised Process
Improvement for Construction Enterprises (SPICE) Model (Sarshar et al. 1998), and the related research area of
learning organizations in construction (Chinowsky et al. 2007).

FIG. 1: A conceptual model for assessment of the industry (from Fayek et al. 2008)
The assessment of the maturity of innovation management practices builds upon previous work on this topic. Willis
and Rankin (2009) have defined a maturity model to assess management practices within the construction industry
at an industry level. The model uses a three level construct for maturity where a practice is: (1) immature, in that it is
ad hoc in its application, (2) transitional mature, in that it is defined and repeatable, and (3) mature, in that it is
measured and improved. The levels correspond to a range from zero to one; zero, where it does not exist, and one,
where it is mature.
2.1 Virtual Reality in Construction
The application of virtual reality technologies and tools in construction has been one of the widely discussed and
researched topics in the construction industry scholars community during the past decade. Many perceive the tools
offered by VR to be very useful in assisting with visualization and in enhancing the understanding of spatial as well
as temporal aspects of the construction process. Those specific advantages of the application of VR tools bring about
overall benefits in the general planning and scheduling of construction projects.
The previous research, completed in the assessment of virtual reality for construction application by many
researchers, has proved that the use of 3D and VR walk-thru technologies could assist in the development of more
complete and accurate schedules through having a significant impact on the schedule review process (Songer et al.
2001). Another area of application includes the use of 4D tools for educating or training purposes. Messner and
Horman (2003) proved through a study that 4D assisted in a better understanding of construction plans, especially
for inexperienced personnel.
Whisker et al. (2003) performed experiments to study the application of 4D in an Immersive Virtual Environment
(IVE). It was shown in the experiments that the use of IVE assisted in reducing the planned schedule duration by
28%, identifying constructability issues and evaluating schedule dependencies. A rather practical study application
was developed for a strategic decision support system for virtual construction, (VIRCON) (Dawood et al. 2005). The
main target of the system was to enhance the ability to trade-off temporal aspects with spatial aspects in order to
come up with a more developed construction schedule. More recently, Dawood and Sikka (2008) studied the
benefits of 3D/4D as a communication tool. Their study and practical experimentation proved that the use of 4D
models is a more efficient tool for communicating and interpreting construction plans compared to traditional 2D
CAD. The study also proved that the findings are valid even amongst experienced personnel who are accustomed to
using 2D CAD.
61
9
th

Thus, the authors have identified four major categories of benefits of VR technology application for the construction
industry. These serve as initial categorizing guidelines for the assessment of the specific VR technology adoption
within a modular construction context. The four areas of possible improvement are: scheduling, space planning,
communication, and training and educational applications.
2.1.1 Scheduling
The development of construction schedules is one of the most comprehensive tasks in construction planning as it
requires the input of a variety of information, as well as involvement of many participants, especially for larger
projects. Due to the importance and complexity of construction scheduling, various efforts have been put into
providing aiding tools and applications and the integration of some of those tools for more efficient scheduling.
Various software applications have been developed to assist in generating more complete and accurate schedules
and to establish formal scheduling approaches. The integration of such software with 3D or 4D visualization has
been studied and is expected to further boost the scheduling capabilities of construction managers. The use of 3D
and VR walk-thru can assist in the creation of more correct schedules (Songer et al. 2001). Correctness includes
three characteristics: completeness (measured by the number of missing activities), valid activity relationships
(measured by errors such as physical impossibilities, redundant or looped relationships) and reasonable activity
duration. Less significantly, but still advantageous, 3D and walk-thru also help to create good schedules (Songer
et al. 2001). Goodness metrics include end date, critical path, and float and resource fluctuation. The use of 4D can
also assist in detecting scheduling logical errors more frequently, faster and with fewer mistakes and can also
compensate for a lack of practical experience (Kang, Anderson and Clayton, 2007).
Studies proved that VR technologies are a valid and comprehensive tool that provides not only a better
understanding of spatial and temporal aspects (hence avoiding errors), but also identifies more areas of
improvement, and boosting confidence in generated schedules.
2.1.2 Space planning
Space is considered one of the more critical resources in construction projects. Due to the fact that many participants
or crafts are involved at any specific time in an average to large size project, the ability to plan for space distribution
is difficult and often spread amongst participants. If effective communication is lacking, this planning spread would
face confusion, unexpected space clashes, delays and increased costs.
Naturally, VR technologies will provide a better understanding of space through visualization. Project participants
can effectively analyze problems regarding sequential and spatial conflicts prior to actual construction operations.
Research has covered this area of VR applications. VIRCON (Dawood et al. 2005) has a wide set of space planning
tools (some produce plans automatically through simulation and databases and others used as aiding tools for
manual planning). These tools include assigning plants and temporary works to space plans, checking for possible
space clashes, marking-up available space and distributing tasks over the life time of the space. The use of 3D
modelling has long been used for plant design to check for space requirements and it is only natural for 3D/4D and
VR technologies to be developed for the analysis and planning of space in actual construction site settings. Better
space planning can lead to valuable benefits in other ways such as in the reduction of overall project duration and in
maintaining good relationships amongst project participants.
2.1.3 Communication
Improving the efficiency of communicating construction plans and schedules could be argued as the essential
advantage and from which other benefits of using VR stem. Visualization, in concept, is the process of displaying
information in order to assist in understanding and evaluating it. Improved communication could lead to time
savings through reduction in reconstruction. This is due to the increased ability to illustrate logic, sequence and
interrelationships among construction tasks and products. Also better communication leads to increased confidence
in all aspects of planning.
The traditional use of 2D designs and plans introduces unnecessary secondary stages and tasks within the planning
process. It can also cause confusion when information is transferred among participants. In addition, the use of
traditional 2D plans requires a variety of information storage media and hence makes the planning process
vulnerable to misinterpretations and loss of information. Although the creation of 3D and 4D models and plans
62
9
th

demands more resources initially, if applied appropriately, these tools could save on the total time and effort for
overall planning (Kang, Anderson and Clayton, 2007).
2.1.4 Training and educational application
The use of VR technologies as a visual communication enhancer can greatly improve the ability to learn and gain
rapid experience related to various construction management and design skills. Advanced visualization tools can
also assist students or trainees by providing them with the chance to assess their decisions and their impact. This
option is not likely to be feasible in an actual site setting as training is mostly observatory due to the high cost of
errors. Even experienced personnel might not be as willing to try a new approach or method due to the conservative
nature of the construction industry. VR, along with simulation, has proved to provide an effective remedy such that
learning is more proactive and less traditional.
Several experiments were conducted by Messner and Horman (2003) to measure the added value for students when
using 4D CAD tools when reviewing construction plans. Several conclusions were drawn: the benefits of using 4D
CAD includes improving the understanding of sequencing issues and their importance, improving the ability to
evaluate important flow concepts, quickly understating complex building models and gaining experience at faster
rates.
2.2 Modular Construction
Recent increasing concentration on aspects such as cost, schedule and labour issues within the construction industry
has made prefabrication, preassembly and modularization in construction more feasible than ever before. In addition
to those drivers, advances in information technologies and construction engineering software would appear to make
pre-work (a term that encompasses the aforementioned three similar construction methodologies) easier to apply and
with reduced accompanying risks.
Modular construction is, by definition, a term that stands for the systematic approach of breaking down a
construction product design into complete systems that are fabricated off-site with the involvement of multiple
trades in a controlled environment, and then transported to the construction site and assembled with minimal effort
(in comparison with their fabrication) (Haas and Fagerlund, 2002). The concept of modular construction is flexible;
the implementation of modular construction strategies and methodologies cover a wide spectrum of applications and
for a variety of applications (Gibb 1999).
There are various drivers and benefits to the application of modular construction. Some of those benefits are easy to
recognize and some vary, depending on the application scenario. The major drivers to the use of modular
construction relate to the general parameters of any construction project which are cost, schedule, quality and safety
(Gibb and Isack 2003). Other secondary, but still significant parameters include the environment, maintenance,
design, secrecy and others. In terms of cost, there are many possible ways through which the application of modular
construction can result in savings. Increased productivity of workers, due to the controlled and more organized
environment, as well as easy access to tools and equipment in modular construction, by nature, reduces project costs.
Another major source of decreased overall costs is the reduced cost of onsite labour. Schedule drivers are also one of
the most significant when making the decision to modularize. A modular context is by nature more repetitive and
includes fewer variables to account for when planning and scheduling projects. This repetitive character as well as
the controlled environment also translates to more efficient and less costly quality control and enhanced safety (Gibb
and Isack 2003).
Modular construction appears to have gained greater consideration during times of high construction demand and
activity industry wide. For example, there was a significant rise in modular residential construction activity during
the economic boom that followed the energy crisis of the 1970s. Looking at the current economic conditions and
the construction industry overall, one can deduce that there will be an opportunity to increase the demand for
modular construction to provide for efficiency gains and the ability to meet infrastructure demands worldwide. Even
as the current conditions stand, modular construction can still be recognized as being on the rise as international
efforts have incorporated it into their industry initiatives and the current NRC Construction Key Sector Group has
identified prefabrication, preassembly and modularization as a new effort in its strategic plan (NRC 2006).
Observing the variety of benefits that could be achieved through modular construction, it could be presumed that its
application should be highly rewarding and attractive. However, there are various implications and issues that hinder
63
9
th

the spread of modular construction within the construction industry worldwide. Modular construction represents a
significantly different approach when compared to traditional onsite construction. This difference raises new issues
and impediments that influence the decisions to undertake modular construction. One area of major concern is
related to engineering requirements. Depending on the extent of pre-work, it may be necessary to complete 90% of
the engineering design prior to construction, as opposed to the 40% generally necessary for conventionally built
projects (Tatum et al. 1987). In addition to the need for early design completion, there are additional factors to be
addressed. The specific dimension or loading limitations, due to transportation constraints, are one of the most
prominent factors in modular construction (Fussell 2007).
Scope flexibility is also expected to be decreased with modular construction. A well defined scope is essential for
effective project planning and to avoid any changes later in the life cycle of the project; note that changes are
significantly more costly in a modular construction scenario when compared to traditional methods. Finally, the
increased demand for effective coordination and communication among participants is a barrier to modular
construction. The distribution of the work load, formation of work breakdown structures, progress monitoring,
scheduling and organizational structures might all need alterations from the traditional sets to provide for a
successful modular construction project. All those alterations require highly effective communication and
collaboration among participants (Prieto 2008).
2.3 Opportunity of VR Application in Modular Construction
Observing the aforementioned impediments of modular construction, it can be concluded that VR application
opportunities exist and can assist in facilitating modular construction. Haas and Fagerlund (2002) and others
recognized the importance of computer integration and technological advances such as 3D CAD and the possibility
of its application in modular construction. VR technologies go beyond the capabilities of 3D design to include
aspects such as time and more effective communication. Also, the application of VR would be facilitated due to the
need to have a significant proportion of the engineering design completed prior to commencing construction
activities. The completed engineering design would provide for all the input to generate useful virtual reality tools
that can be applied to enhance performance.
Enhancing physical interface management could be one of the most direct and initial advantages of using virtual
reality technologies. The visualization enhancement provided by VR technologies would assist the engineers in
assessing the complex modules and plan for efficient assemblies in terms of fabrication as well as installation.
Another aspect that can be improved is spatial planning for transportation concerns. While trading off with temporal
aspects, engineers can implement virtual reality technologies to generate efficient transportation schedules and
plans.
Mitigating the reduced scope flexibility could also be another advantage of VR application. VR allows for effective
communication of plans with other project participants during the early stages. This would allow owners to have a
better idea of the end product and perhaps suggest alterations prior to the initiation of work. This would also
increase confidence and improve the relationship with the owner. Increasing the efficiency of communication will
also benefit the overall project at which coordination of multiple sites is needed. The distribution of information
among participants can be enhanced and also VR technologies can provide the interface for drawings and
visualizations needed in modular construction projects, in addition to traditional plans.
3. INITIAL CASE STUDY FINDINGS
The VR technologies to be implemented are developed in partnership with the NRC-CCCT. In addition, two
industry participants were identified and secured for the practical application of the VR technologies and adoption
assessment. The two companies offer a unique perspective on varying prefabrication technologies and applications
which should assist in having an unbiased and more comprehensive understanding of the prefabrication industry and
its technology challenges. Following meetings with the industry participants to initially identify the direction and
general needs and challenges within the prefabrication construction industry, further analysis was performed to
identify which VR technologies are available and match the needs and resources of the industry. The limitations of
VR technologies were identified, as well as the challenges to their practical application.
64
9
th

3.1 Identified Challenges
Industry practitioners in modular construction were identified through a session organized by the NRC Institute for
Research in Construction to explore the creation of a national network for modular construction (NRC 2009). The
two companies were selected primarily due to the diversity of their operations, from several perspectives. The first
company specializes in mass production of wooden wall panels. The company utilizes an imported system which is
mostly automated for the fabrication of specifically designed wall and floor panel sections. The company fabricates
wall panels for large housing projects and was initially a traditional general contractor which decided to make the
transition to prefabrication about a year ago. The second company specializes in the production of composite wall
panels. The panels are fabricated using a patented design utilizing both concrete and cold rolled steel. The company
aims more at securing larger projects which provide for more feasible investment. Originally, the company
specialized in manufacturing machinery used for fabrication of steel office supplies. Following a decline in the
manufacturing industry, the company decided to go into the construction business and apply some of the
manufacturing principles to gain an advantage. The company still attempts to manufacture their own machinery for
specific tasks such as welding, whenever possible, although that is limited to resources and investment feasibility
(size of projects). Table 1 provides an overview of each company, illustrating the unique and different perspective
each offers.
TABLE 1: An overview of the case study participants.
Factor Company A Company B
Location Southern Ontario Southern Ontario
Company Background Construction (general contractor) background Manufacturing background
Product Wooden Panels Composite Panels (Concrete/Steel)
Main Market Housing Industrial
Source of Fabrication System Purchased In house developed / Patented
Scripted interviews were prepared with open-ended questions to collect data on the opportunities for VR previously
identified and the general structure and operations of each organization. The visits consisted of the interview with
senior management and a tour of the facilities. The results were then summarized and validated with the interview
participants. Each company offers a unique view of prefabrication in construction, where not only their processes
but also their approach towards prefabrication is rather different. Nevertheless, both identified the same two major
issues as their main challenges: communication and integration.
3.1.1 Communication
General communication and education issues with multiple players within the industry have been raised as a general
concern. Modular construction is relatively new in Canada as a significantly different and more progressive
methodology when compared to Europe, for example. This causes clients, architects and subcontractors to handle
modular construction, even in its most simple forms, with scepticism and worries of increased complexity in
assembly and reduced flexibility such as aesthetic options. Enhanced, more effective and practical communication is
needed to establish confidence within the industry and to educate the industry on the use and advantages of modular
construction. Some of the communication challenges that were identified were: ability to communicate the available
options to the architects and clients and the ability to communicate the assembly processes or functionality of the
end product to the architect, client and more importantly subcontractors who will be undertaking assembly on the
site. The communication issues identified were more directly related to opportunities for VR application tools to
enhance the efficiency of communication, and in some ways, promote the methods of prefabrication through
educating the participants.
3.1.2 Integration
Although not initially included in the scripted interview, the issue of integration was identified as one of the
significant issues causing delays within the production processes. Also, integration issues and complications were
found to exist on two major fronts: integration between architectural drawings and construction designers/detailers,
and the integration between completed design and the machinery used for the manufacturing of designed assemblies.
Existing integration issues are mainly an underlying form of interoperability. Interoperability is a varying and
widely discussed topic with continuing long term efforts to resolve case-specific interoperability issues. The general
approach to resolving interoperability issues, which was the same followed by the selected industry participants, is
65
9
th

to use an off-the-shelf application or software or one that is developed by a third party specializing in software. The
companies then attempt to standardize the software within their systems in order to limit any integration issues.
However, interoperability or integration issues and problems have been found to always exist and are rarely totally
remedied. Integration still is one of the major bottlenecks within production processes for two major reasons: the
involvement of various participants, making it difficult to have one standardized approach adopted by all involved.
Second, because of the fact that the prefabrication industrys demand for integration solutions is limited in volume,
few software developers are willing to offer efficient and tailored solutions for their integration problems, and even
those offered are charged in total to the single specific order, which makes them rather costly.
3.2 Virtual Reality Technologies Solution
Following the identification of the main challenges faced by the industry participants, the focus was then on
assessing the VR technologies available, their limitations, and how they could be tailored to fit the needs of the
industry. An overall assessment of the NRC-CCCT facilities in London, Ontario was conducted in order to evaluate
its VR technology capabilities. The VR facilities include state of the art environments and hardware such as
separately functioning theatre, cave, and motion capture environments, as well as other more portable options such
as the 3D scanner, LCDs and compact and powerful processors. In addition, the VR tools and technologies
available have been implemented in various manufacturing as well as construction-oriented applications. Alongside
sophisticated modelling and visualization for the auto manufacturing industry, examples of previous projects include
training environments for crane operators and motion capture of human/environment interactivity. The following is
a summary of the significant aspects of the available VR technologies related to the need identified:
The use of portable VR hardware has very few and case-specific limitations which are not likely to exist
within the scope of the usage scenarios to be applied.
In terms of software, various programs are available for use in multiple applications such as basic 3D
design software, more advanced graphics software, animation and interactivity software, as well as in-
house programmed software for specific applications.
Software programs can be integrated and combined for a more complete and practical usage scenario.
Previous work on creating a VR model for a basic demonstrative construction application has been done
and the findings of the project (in terms of resources required and effort) are considered.
It was identified that in order to have a real VR application, interactivity was a vital element of the VR
tools to be offered to the industry participants.
In order to have a feasible application, it is necessary to have tools that can be used on a continuous basis.
For example, directories of components that can be repeatedly used instead of single, custom or project-
specific applications.
Therefore, taking into consideration the needs identified and the capabilities of the technology, the main focus of the
project is on the application of VR technologies as communication process technologies and tools with all types of
participants. Minor assessment and consideration of integration issues will still be considered, however, integration
issues would be addressed strategically rather than technically. The application of the VR technologies will include
interactive elements and not only advanced illustration tools. The creation of reusable tools is an essential aspect and
that can be accomplished through creating directories of objects or tools which facilitate continuing use of the VR
technologies rather than keeping it case-specific.
4. NEXT STEPS
The immediate next steps in the research project consist of developing the VR technology prototype and assessing
the organizations capacity for implementation and adoption through an assessment of innovation practices. The
development of the appropriate VR technologies is relatively straightforward with the assistance of NRC-CCCT
expertise. The assessment of an organizations capacity for innovation requires the completion of an appropriate
framework.
66
9
th

4.1 Developing a Technology Solution
In order to establish an adequate technological environment suited for the practical application of the VR
technologies by the industry participants, four major steps need to be completed: 1) gather additional information
and feedback regarding the specific needs of the participants and their use of the VR tools, 2) acquire all detailed
design information from the participants needed to create the VR environment, 3) start with the creation of the basic
3D models and components of the VR environment, and 4) complete the VR tools by introducing interactivity
elements to the 3D environment, as well as adding the element of time for the creation of 4D models.
There are various options available for the creation of the 3D components in terms of software. On the other hand,
the use of hardware such as the 3D scanner is not likely due to the difficulty of establishing smart models using
these tools (models that include groupings, components, hierarchies etc.). The NRC-CCCT facilities and
technical support enables importing 3D models made using any of a number of software options such as 3D CAD,
Maya, 3D Max and Google SketchUp. Due to the experience of the researcher with Google SketchUp, the relative
simplicity of the construction models, when compared to other manufacturing modelling, and the free availability of
the software (for both model creator and end user), Google SketchUp was selected to create the 3D environment.
However, the models created with Google SketchUp will be enhanced graphically using Maya at a later date before
incorporating interactivity and time elements.
4.2 Innovation Maturity Framework
Figure 2 depicts a conceptual framework of how innovation management practices will be assessed for the case
study companies. Innovation management has been broken down into factors that influence its outcome. The factors
will be grouped and each grouping is to be assessed based on the plan-do-check-act management cycle. The
assessment will be completed through a series of questionnaires structured to determine the level of maturity within
an organization. The maturity is reported with respect to the level achieved and remaining for improvement for each
management cycle step (e.g., bar chart) and can also be compared against other organizations in a benchmarking
exercise (e.g., radar chart).

FIG. 2: Conceptual framework for assessing the maturity of innovation management processes.
67
9
th

The case studies are being used as a step in validating the factors and groupings. A weighting of factors will then be
completed based on pair-wise comparisons by employing the analytic hierarchy process, where each step within a
grouping is weighted and then each grouping of factors is weighted. When completed for a group of experts, the
geometric mean of the results will be used to determine the contribution to the maturity scores. This allows for
analyses as presented in Figure 2. The chart is indicating the relative importance associated with each step within
innovation management along with a maturity score (achieved) and opportunity for improvement (remaining) at an
organizational level. This will then give a comparison with the level of success in implementing VR (performance
impact), however, a case study of two will not be significant, therefore a broader study will be conducted.
The paper is reporting on work in progress that is intended to support the development of a means of assessing the
innovation management practices of construction industry organizations. The results will be used to identify
correlations with performance and as a predictive tool for implementation and adoption of process technologies.
5. ACKNOWLEDGEMENTS
The authors wish to express their thanks to the case study participants for their time dedicated to this research
project. Appreciation is also extended to the National Research Council of Canadas Centre for Computer-assisted
Construction Technologies and Institute for Information Technology who are supporting the research. Sponsorship
is also provided by the Natural Science and Engineering Research Council of Canada.
6. REFERENCES
Andersen, E.S. and Jessen, S.A. (2003). Project maturity in organisations. International Journal of Project
Management, Vol. 21, 457-461.
Chinowsky, P., Molenaar, K. and Realph, A. (2007). Learning organizations in construction. Journal of
Management in Engineering, Vol. 23, No. 1, (January), 27-34.
Crosby, P. B. (1979). Quality is Free: The Art of Making Quality Certain. McGraw-Hill Book Company. New York
Dawood N., Sikka S. (2008). Measuring the effectiveness of 4D planning as a valuable communication Tool,
Proceedings of the 7th International Conference on Construction Applications of Virtual Reality,
Pennsylvania State University, USA.
Dawood N., Scott D., Sriprasert E., Mallasi Z. (2005). The virtual construction site (VIRCON) tools: an industrial
evaluation. ITcon, Vol. 10, 43-54.
Fayek, A., Hass, C., Rankin, J., and Russell, A. (2008). Benchmarking program overview: CSC performance and
productivity benchmarking program, phase 1, CSC-CBR Technical Report P1-T12, Ottawa, Ontario, Canada,
23 pp.
Finnemore, M., Sarshar, M. & Haigh, R. (2000). Case studies in construction process improvement. Proceedings of
the Arcom Construction Process Workshop, Loughborough University, Loughborough.
Fussell, T. et al. (2007). Off-site manufacture in Australia: current state and future directions. A report for the
Cooperative Research Centre for Construction Innovation, Queensland University of Technology, 51 pp.
Gibb, A. G. F. (1999) Off-site fabrication: prefabrication, pre-assembly and modularisation. John Wiley and Sons,
ISBN 0470378360.
Gibb, A. and Isack, F. (2003). Re-engineering through pre-assembly: client expectations and drivers. Building
Research and Information, Vol. 31, No. 2, 146-160.
Haas C. T., Fagerlund W. R. (2002). Preliminary research on prefabrication, pre-Assembly, modularization and off-
site fabrication in construction. A Report to the Construction Industry Institute, The University of Texas at
Austin, Austin, Texas, USA, 76 pp.
Haas C. T., OConnor J. T., Tucker R. L., Eickmann J. A., Fagerlund W. R. (2000). Prefabrication and preassembly
trends and effects on the construction workforce. A Report of Center for Construction Industry Studies,
Report No. 14, The University of Texas at Austin, Austin, Texas, USA.
68
9
th

Kang J., Anderson S. and Clayton M. (2007). Empirical study on the merit of web-based 4D visualization in
collaborative construction planning and scheduling, Journal of Construction Engineering and Management,
ASCE, June 2007, 447-461.
Kwak, Y.H. and Ibbs, C.W. (2002). Project management process maturity (PM)2 model, Journal of Management
in Engineering, Vol. 18, No. 3, (July), 150-155.
Messner J. and Horman M. (2003). Using advanced visualization tools to improve construction education,
Proceedings of the3rd International Conference on Construction Applications of Virtual Reality, Virginia
Tech, USA.
NRC (2009). Industrialization in building construction: report on the industry stakeholder meeting, Ottawa, April
24-25, compiled by Suthey Holler, National Research Council.
NRC (2006). National research council strategy 2006-2011, 28 pp.
Paulk, M., Weber, C., Curtis, B. & Mary-Beth, C. (1995). The capability maturity model: guidelines for improving
the software process. Addison-Wesley Longman Inc. USA.
Prieto, R. (2008). A great LEAP: modularization as a strategy, The Voice, The Official Magazine of the
Construction Users Roundtable, Fall, 47-52.
Sarshar, M., Hutchinson, A., Aouad, G., Barrett, P., Minnikin, J. and Shelley, C. (1998). Standardised process
improvement for construction enterprises (SPICE), Proceedings of 2nd European Conference on Product and
Process Modelling, Watford, UK.
Songer D.A., Diekmann J., Rasheed K. and Hays B. (2001). Construction scheduling using 3-D CAD and walkthru,
Construction Innovation, Vol. 1, 191-207.
Songer D.A., Diekmann. and Karet D. (2001). Animation-based construction schedule review, Construction
Innovation, Vol. 1, 181-190.
Tatum, C. B., Vanegas, J. A., and Williams, J. M. (1987). Constructability improvement using prefabrication,
preassembly, and modularization, Construction Industry Institute, The University of Texas at Austin.
Tooles, T.M. (1998). Uncertainty and home builders adoption of technological innovation, Journal of Construction
Engineering and Management, ASCE, Vol. 124, No. 4, 323-332.
Whisker V., Baratta A., Yerrapathruni S., Messner J., Shaw T., Warren M, and Rotthoff, E (2003). Using immersive
virtual environments to develop and visualize construction schedules for advanced nuclear power plants.
Proceedings of ICAPP '03, Cordoba, Spain, Paper 3271.
Willis, C. and Rankin, J. (2009). Maturity modeling in construction: introducing the construction industry macro
maturity model (CIM3), CSCE 2nd International Construction Specialty Conference, St. Johns
Newfoundland, May, 10 pp.
69
9
th

COLLABORATIVE 4D REVIEW THROUGH THE USE OF
INTERACTIVE WORKSPACES
Robert M. Leicht, Ph.D., BIM Manager,
DPR Construction;
robl@dprinc.com
John I. Messner, Ph.D., Associate Professor of Architectural Engineering,
The Pennsylvania State University;
jmessner@engr.psu.edu and http://www.engr.psu.edu/ae/messner/
ABSTRACT: This paper presents quasi-experiments conducted during the validation process for a framework to
plan interactive workspaces titled INVOLVE. The validation is focused on the value demonstrated for different
interactive workspace components, focusing on interaction with the model, while teams perform a 4D schedule
review task. Interactive workspaces are spaces with potential for ubiquitous electronic interaction to allow for
enhanced communication and capture of information. In the experiment reported, 16 groups of students, with
typically 3 group members, were asked to review the 4D schedule of the Long and Foster Headquarters project in
Chantilly, VA. The 4D schedule reviewed during the task was a modified version of the contractors actual project
schedule to incorporate a total of 11 known schedule challenges. The focus of the study was to identify changes in
the students interactions when using an interactive workspace, and the differences that occur when model
interaction was limited through the interface or through the sharing of the interactive device and display. This
paper will provide results on the students perceptions of the value of the workspace and interaction through a post-
test questionnaire.
KEYWORDS: Interactive Workspaces, 4D Modeling, Interaction.
1. INTRODUCTION
Collaboration plays a central role in the design and construction of buildings, infrastructure and other facilities.
Recent developments in technology, such as building information modeling, virtual reality, and computer mediated
communication, have been shown to add value to this collaborative process. Previous work into virtual
environments (VE) has found that benefits from design and construction are focused on team design support and
decision making tasks (Gopinath, 2004). Despite this value, it is challenging to document the traits which are
essential in virtual environments and how these contribute to improved team collaboration. Defining these benefits
as they relate to specific characteristics provides a means for comparison between different VE setups, interactions,
and applications. To these ends we have undertaken a study to compare between two different variables within an
interactive workspace to study how teams value the use of the different tools which are available.
The study presented in this paper represents a step in the validation process for the INVOLVE Framework. The
INVOLVE Framework is intended to allow for the planning of virtual prototype use for the alignment of the
physical media with the needs of the virtual content and task for improved collaboration (Leicht, 2009). INVOLVE
has seven aspects which are interrelated for the collaborative and communicative aspects of using virtual prototypes:
I Interaction the physical mode of interaction between the user and the system
N Network the computing infrastructure to connect the system with internal and external systems
V Virtual Prototype digital representation of real world items for interaction, simulation, and testing
O Organization the team, the project structure, and the competencies needed to complete the task
L Layout the physical space and elements needed for teams to work together for a task
V Visual Display the location, resolution, and type of display system utilized to share information
E Existential Collaboration Authentic collaboration through shared goals, knowledge, and resources
71
9
th

The two traits of the interactive workspace to be studied are the visual display ability of more than one user to be
able to work in parallel to add and display information, and the value of having and virtual prototype interaction in
the form of the 3D model for a 4D schedule review.
2. USE OF INTERACTIVE WORKSPACES
The available literature offers evidence of improved performance when teams meet and work using improved
visualization tools and software within interactive workspaces. Fruchter (2005) studied a group of industry
members given a teams project at a set point to discuss and generate ideas for future directions and challenges while
the students observe and ask questions. In the findings, Fruchter identified the unique aspect that the virtual
attendees are actually more engaged in the content due to their proximity to the developed material than the physical
attendees. Wang et al (2007) performed a comparative observational study to discern the role that software played
when using a virtual environment for team problem solving for a construction scheduling task, finding greater
enjoyment and interest from the student when meeting in an interactive workspace. Issa et al (2007) found added
value for student design projects when using the Interactive Collaboration Lab at the University of New Brunswick.
Students demonstrated that the lab enabled them to collaborate more effectively, make more educated decisions,
more effectively use their time, and produce higher quality work. These studies demonstrate added engagement and
greater value derived from meetings using interactive workspaces, but the specific impacts of hardware and interface
considerations are not the focus.
Some of these efforts have proven beneficial in field construction settings as well. Liston compared traditional
construction project meetings with project meetings using 4D visualization and identified increased efficiencies in
communications among team members (Liston et al, 2001). Khanzode found improved coordination among
specialty contractors on the Camino Hospital Project when the team developing the shop drawings for the duct,
piping, plumbing, and sprinkler systems were housed in the Big Room there was a shorter time and more
economic process for collaborating. (Khanzode et al, 2008). While these studies demonstrate the value which can
be derived from the visualization and teaming aspects created in interactive workspaces, it is still not clear which
traits of the systems are adding value, the value is considered in the context of meetings and not for performance of
differing tasks.
3. THE IMMERSIVE CONSTRUCTION LAB
The interactive workspace utilized for this study was the Immersive Construction (ICon) Lab at Penn State. The
ICon Lab was developed in stages starting with a single rear-projected screen. Currently, the ICon Lab has three 6
feet (1.8m) tall by 8 feet (2.4m) wide screens, with the two side screens angled at 30 degrees from parallel from the
center screen as shown in Figure 1. All three screens are rear projected, with two projectors for each to allow for
passive stereo visualization. The display is run from a single Windows-based desktop computer.
In addition to the central display, a single interactive whiteboard is set up adjacent to the left side of the main
display. The interactive whiteboard is connected to a separate computer from the main display. Along with the
interactive whiteboard a set of 20 tablet PCs are housed within the ICon Lab. The tablets are set up to allow the
users to wirelessly push the screen image to one of the three screens of the central display.

72
9
th

Figure 1: Images of teams using different lab configurations and a rendering of the Immersive Construction Lab at
Penn State.
4. RESEARCH METHODOLOGY
The undertaken analysis is an intermediate step in an ongoing study to validate the INVOLVE framework for
utilizing interactive workspaces more effectively. The validation process utilized a series of observational quasi-
experiments for different tasks in conjunction with questionnaires to gain insight into the participants perspectives
of the media use for the four different types of tasks identified for use with the INVOLVE Framework: Create,
Integrate, Examine, and Focus. The 4D review task was chosen as a means of evaluating the interaction and display
parameters influencing an examination task. In this analysis, an undergraduate class in architectural engineering at
Penn State performed a 4D review activity in the ICon lab as part of a semester long project. An examine task is
defined as: Reviewing the design/solution for viability and to ensure compliance with the project goals, code
requirements, and owner and design intent (Leicht, 2009). The selected media elements were chosen as part of the
ongoing framework validation to help identify how the interaction with the 3D model and the use of shared
interaction with the display might impact the collaboration amongst the teams.
The project chosen for the 4D review was part of a team project, the student teams performed estimates, scheduling,
and site utilization planning for the Long and Foster Headquarters building, located in Chantilly, VA. Of the 95
students in the class, 16 groups of volunteers were used with typically 3 members each. The class was very
homogeneous, with the class admission limited to students in architectural engineering with 6
th
semester standing,
the student body is aged 20-22 years old for over 95% of the students, approximately 80% male, with 35% having
some limited design or construction experience. The 4D review task was held during the 12
th
week of a 16 week
semester. The task for the students while in the lab was to review the 4D simulation and identify schedule conflicts.
The 16 groups were randomly assigned to one of four different arrangements within the lab. The arrangements were
set up around 2 central variables, the number of computers for interaction and navigability of the 4D model.
When the students arrived for the activity they were provided with a brief overview of the lab layout and the tools
available, then they were then given 45 minutes to review the 4D simulation. The students then submitted both
individual and a team list of issues. At the end of the period, the students were asked to rate 20 statements about the
utility and value of the lab on a 1-5 Likert Scale. The teams were asked to perform the task in the lab as an
opportunity for extra credit, with the option of submitting the questionnaire. The students had an incentive to
perform well with the results on the conflicts identified related to the extra credit they received, but the professor
and teaching assistant were not present for the activities and did not see the questionnaire results until after grades
were submitted for students.
Using the INVOLVE framework, the teams were divided into four treatments with four teams randomly chosen for
each of the treatments. The 4D model was developed with a list of known errors, potential areas for improvement,
and potential safety concerns. The model was exported into a video file, with eight groups provided with the
Navisworks 4D model and the other eight having the exported video, thus creating the variable for the interaction
with the 3D model. In regard to the display interaction, eight groups were provided with a central display with a
73
9
th

mouse and keyboard for operating the simulation, the other eight teams were provided with three laptops with the
displays shared on three large screens.
As shown in Table 1, there were four different configurations of the ICon Lab based on the traits of the interactive
workspace. In these configurations, the differing displays influenced the availability of interfaces for team members
to draw upon, while the software interaction was based on the availability of navigating the 3D model while
reviewing the 4D simulations. Layout III should prove the best layout for the task with a shared central display and
greater interaction with the 4D model.

Table 1: Matrix showing the breakdown of the four layouts used for the study.

While the large class does provide a large sample for study, by breaking the 16 groups into 4 configurations, the
subsets for each layout are not large. Also, because the feedback is based on the questionnaire responses the results
are based on subjective evaluations of the team experience, and may be biased. Particular bias may result from the
students having limited exposure the ICon Lab creating a small novelty effect, though that should hold true for the
whole sample. Also, since the participants were all students there was little to no construction experience which
means the process and discussions were less efficient and there may have been an added level of difficulty for the
task when putting the students into a new environment to perform the review.
5. RESULTS AND DISCUSSION
There were four areas on which feedback was collected from the participants. With the small sample sizes for the
four different layouts, little was expected in terms of statistical significance. The intent was to identify if the
perceptions of the value for the displays aligned with the expectations for the variables. The evaluation of the value
of the displays was handled through observational studies of the group which are not reported within here.
To begin the questionnaire regarding the experience of performing the 4D review task in the ICon lab, the focus was
on overall feedback. The teams consistently enjoyed the experience with all four Layout average close to 4.0 out of
5.0 in agreement. The second question asked about the suitability of the ICon lab for the 4D review task. While all
74
9
th

of the layouts averaged above 4.0, in agreement that it was well suited, the disparity among the four layouts align
with the expected variables which contribute to the use of Interactive Workspaces for such a task. The layouts
which had the strongest agreement were the teams which had a central display and use of the model for navigation
and changing viewpoints. For the last two questions there was also general agreement among all of the layouts that
the individual members were involved in the process and that they would be likely to use the lab or a similar space
for this type of task in the future.
Figure 2: General feedback from groups regarding their experience using the ICon Lab for a 4D review
task.
*The statements listed on the X-axis have a scale of 1 (Strongly Disagree) to 5 (Strongly Agree) on the Y-axis.

5.1 Feedback regarding the display interaction
After reviewing general items about how the team worked, the questionnaire considered the interaction with the
display amongst the four layouts. The first question focused on the value of the large display system. The teams
which used only the large display found it very useful with an average above 4.0 in agreement. The teams which
had laptops were just above neutral, with one participant explaining:
I tended to look at the computer screen instead of looking at the big screens. I feel that using the
bigger screen is better but because I had to use the mouse to start and stop the simulation it was
just natural for me to look at the smaller computer screen.
The second question suggested that there was little value to having multiple screens. The average across the four
layouts suggests a slight preference for multiple screens, but not much. The students using the lab Layout which
used the large display and the video agreed slightly with the comment that the extra screens were unnecessary. The
last two statements that the environment allowed the team to communicate effectively and for all members to
contribute were consistently agreed to above a 4.0 level by all layouts.
75
9
th

Figure 3: Feedback regarding the value of the display system.

5.2 Model Interaction Feedback
When asked about the value of the interaction available when viewing the 4D simulation, the responses became
more disparate. The teams which had open access to navigate the model felt that they were able to view the
necessary perspectives within the model, with agreement above 4.0 for both layouts which had the model. The
layouts which had the video file of the simulation slightly disagreed, with both averaging below a 3.0. When asked
if the teams shared control of the model and display, the teams with laptops all agreed above a 4.0 level, the teams
with the model and central display agreed somewhat (3.5) and the teams with the video file and central display came
out at a neutral level indicating a notable disparity in the perception of the groups about their ability to use the
virtual prototype. All four layouts agreed that the model was useful for communicating information about the
schedule issues above a 4.0 level. When asked if there was a single primary user of the model and display, again
Layout IV was neutral, and Layout II felt that there was no primary user, with Layout I and III close to neutral but
suggesting that some teams had a primary user.

76
9
th

Figure 4: Feedback regarding the value of model interaction.
*The statements listed on the X-axis and a scale of 1 (Strongly Disagree) to 5 (Strongly Agree) on the Y-axis.

5.3 Collaboration Feedback
The last area for feedback was focused on the impact of the layout on the collaborative effort of the team.
Team members in all four layouts felt that the teams worked productively during the task, with an average
score above a 4.0 level, and most reaching 4.5. When asked if their team collaborated better than their
project team, most teams were neutral or slightly above. Team members in all layouts felt that everyone in
the group contributed ideas and suggestions during the task with an average score above a 4.0 level. When
asked if it was challenging to reach a consensus on the issues in the schedule, the layout responses varied
by notable amounts. Layout I and IV felt that it was relatively easy to reach a consensus, while Layout II
and Layout III closer to being neutral, again creating a disparity by layouts.

Figure 5: Feedback regarding the collaboration which took place in each team.
What stands out is how closely the responses regarding meeting a consensus reflect the responses about how well
suited the lab was for the 4D review task. Layout III had the highest rating for the lab being suitable for 4D review
and found reaching consensus the easiest among the four layouts. As suggested earlier, Layout III was identified as
77
9
th

ideal for the task. Reaching a consensus implies getting the team members to understand the issue and all agree that
it is a problem; both the individual student perception feedback and the quantitative evaluations of the outcomes
show that this configuration was the best for teams to reach consensus for this type of task.
6. CONCLUSIONS AND FUTURE WORK
This paper presented questionnaire results for teams utilizing an interactive workspace for a 4D review task. The
use of the Interactive Workspace for the 4D schedule review task was planned to focus upon the shared interaction,
the shared display, and the software capabilities for manipulating the virtual prototype. The mouse and keyboard
interface were the chosen forms of interaction, with the availability of separate laptops allowing select teams to
review the schedule individually first. The assumption of Layout III as the best layout was encouraged by the
feedback as the highest rating for suitability for the task and for ease of reaching consensus.
The outcome from the 4D review suggests that the choice of the interaction and the display relate to the ability of the
team members to communicate for a task. The value in planning the media for this example can be seen in the
feedback from the participants suggesting that the media indicated when using the INVOLVE Framework aligned
with the intended results for an Examine Task based on the perceptions of the participants. In addition, the teams
found the experience in the lab more enjoyable than typical meetings and would again use the interactive workspace
for performing similar tasks in the future.
Future work in this area will explore more thorough analysis of the displayed value of the systems when
collaborating through observational studies of these tasks. The study will also explore other tasks that may find
value in utilizing interactive workspaces. Also, studies of the value of different forms of interaction depending on
the task at hand and interests of the team members involved.
7. ACKNOWLEDGEMENTS
We would like to thank the student participants for their involvement during the study presented in this paper. We
also thank the National Science Foundation for their support of this research through Grant No. 0348457 and
0342861. Any opinions, findings, conclusions, or recommendations are those of the authors and do not reflect those
of the National Science Foundation or the project participants.
8. REFERENCES
Air Force (1994). Glossary, Proceedings of the NSIA Spacecast 2020 Symposium, 9-10 November 1994,
Washington, DC, USA. http://www.fas.org/spp/military/docops/usaf/2020/app-v.htm
Fruchter, R. (2005). Degrees of engagement in interactive workspaces, AI & Society, 19, pp. 8-21.
Gopinath, R., and Messner, J. I. (2004). "Applying immersive virtual facility prototyping in the AEC industry."
Proceedings of the 4
th
International Conference of Construction Applications of Virtual Reality, Lisbon,
Portugal, Sept. 14-15, 79-86.
Issa, M., Rankin, J., Christian, J., and Pemberton, E. (2007). Using interactive workspaces for team design project
meetings. CONVR 2007: 7th Conference of Construction Applications of Virtual Reality, State College, PA,
USA, Oct. 22-28.
Khanzode A., Fischer M., and Reed D. (2008). Benefits and lessons learned of implementing building virtual design
and construction technologies for coordination of mechanical, electrical, and plumbing systems on a large
healthcare project, Journal of Information Technology in Construction, Vol. 13, Special Issue: Case studies of
BIM use, pp. 324-342.
Leicht, R. M. (2009). A framework for planning effective collaboration using interactive workspaces. Ph.D. Thesis,
The Pennsylvania State University, University Park, PA, USA.
Leicht, R., Maldovan, K. and Messner, J. (2007). A framework to analyze the effectiveness of team interactions in
virtual environments, Proceedings of the 7
th
International Conference on Construction Applications of
Virtual Reality, University Park, PA, USA, October 22-23.
Liston K., Fischer M., and Winograd T. (2001). Focused sharing of information for multi-disciplinary decision
making by project teams. Journal of Information Technology in Construction, 6, pp. 69-82.
Wang L., Messner J., and Leicht R. (2007) Assessment of 4D modeling for schedule visualization in construction
engineering education, Proceedings of 24
th
CIB w78 conference on information technology in construction,
June 26-29, 2007. Maribor, Slovenia.
78
9
th

DESIGN SCENARIOS: METHODOLOGY FOR REQUIREMENTS
DRIVEN PARAMETRIC MODELING OF HIGH-RISES
Victor Gane, PhD Candidate,
CIFE, Stanford University;
vgane@stanford.edu; http://stanford.edu/~vgane
John Haymaker, PhD, AIA, LEED AP, Assistant Professor
CIFE, Stanford University;
haymaker@stanford.edu; http://stanford.edu/~haymaker
ABSTRACT:
This paper introduces a collaborative, parametric performance-based design methodology that enables teams to
systematically generate and analyze high-rise building design spaces based on multi-stakeholder requirements.
Building design involves investigating multidisciplinary design spaces with a high number of project-specific
variables and constraints. In practice at leading architecture firms today, conceptual design methods support
generating very few options that respond to a limited number of design requirements. As a result, potentially better
performing design solutions are overlooked. Our research synthesizes a novel, collaborative design methodology
called Design Scenarios (DS). The methodology consists of five process steps: (1) Requirements Model used by a
multidisciplinary team to collect, weigh and prioritize multi-stakeholder requirements, (2) Design Strategy used to
formally transform into parametric models the identified requirements by proposing potential enabling design
parameters and identifying conflicting and enabling relationships amongst requirements and design parameters, (3)
Parametric Process Model used to generate, manage and communicate the complex structure of a resultant
parametric product model from these relationships; (4) Parametric Model used to generate design spaces
responsive to identified requirements, (5) Decisions Model used to support the consensus-building and
documentation of the best decision by visually reporting the design options' performance back to the designers and
stakeholders. We applied DS on a case study presented in this paper. The research is unique in its development of a
method to formally generate parametric models from requirements, and for its industrial-scale, practice-based
integration and testing of formal design and decision making methodologies for high-rise building design.
Improvements are anticipated both in the quality of the design process by reducing uncertainty and inefficiency, and
in the resulting product by enabling more options to be considered from more perspectives.
KEYWORDS: Design space, parametric modelling, process modelling, requirements engineering.
1. OBSERVED PROBLEM
The market economy requires project teams to design quickly and cheaply; however, research shows that successful
design is largely a function of clear definition of end-user requirements (Rolland, 2005) and the generation and
multidisciplinary analyses of a large quantity of options (Kelley 2006). Every project comes up against an inevitable
tension between design exploration and process efficiency. Take high-rise design for example. We recently
conducted a benchmarking survey of existing conceptual high-rise design practice to determine the performance of
leading design teams. We found that on average a multidisciplinary team averaging 12 people can normally produce
only 3 design options during a design process that lasts on average 5 weeks. Most of this time is spent by architects
on generating and presenting a small number of design options. Little time is dedicated to establishing /
understanding project goals and running multidisciplinary analysis. These analyses are inconsistent and primarily
governed by architectural rather than multidisciplinary criteria (Gane & Haymaker 2008). Better performing designs
are likely left undiscovered.

How can high rise building project teams improve design and critical thinking? Understanding and efficiently
managing multidisciplinary requirements early in the design process is a major challenge. So is translating these
requirements into a wide range of design options that designers can quickly analyze and systematically choose from.
Several points of departure partially address these issues. Design Theory helps us understand the general process of
design and define strategies to search the design space. Process modeling can help represent and measure goal-
driven design processes. Requirements engineering can help design teams define and manage their building design
79
9
th

criteria in terms of formally structured goals and constraints. Parametric modeling can help efficiently generate
geometric options. High-rise Design Methods help categorize the types of high-rises and elicit a list of design
constraints, criteria and performance metrics that each category entails (we summarize these in Gane & Haymaker,
2008).

Even with these theories and methods, our benchmarking study shows that the Architecture Engineering
Construction (AEC) industry still lacks a methodology that enables project teams to efficiently integrate them into
practice. They lack a methodology to define and prioritize requirements, translate these requirements into
geometrically flexible parametric models, to analyze these models efficiently from multiple perspectives, and to
understand the multidisciplinary tradeoffs of individual options and spaces of options. In another paper we describe
how the lack of such a method substantially reduces the effectiveness of parametric methods and stalls
multidisciplinary design and decision making processes (Gane & Haymaker 2007). This research establishes such a
methodology and begins to test its impact in practice.
2. THEORETICAL POINTS OF DEPARTURE
In this section we describe the fundamental points of departure for this research.
2.1 Design theory
Design is a creative process, where part of the task is to formulate the problem itself (Simon 1969). Design teams
are aided by multi-stakeholder value-based design and decision making methodologies (Lewis et al 2007, Jin &
Danesh 2007, Keeney & von Winterfeldt 2007). AEC focused researchers are developing related theory and
methodologies, describing the design as (1) identifying a set of requirements; (2) prioritizing among these
requirements; (3) developing preliminary solutions; (4) evaluating solutions; (5) establishing final design
requirements, preferences and evaluation criteria (Akin 2001). Others are applying these concepts in formal design
and decision making methodologies (Ellis et al 2006, Haymaker & Chachere 2007).

While designing, teams construct a design space, formulated as the sum of the problem space, solution space, and
design process (Krishnamurti 2006). Two prevailing strategies emerge to describe the process of constructing a
design space: breadth first, depth next or depth first, little breadth. Designers typically consider a very small
number of alternatives as a result of cognitive limits (Woodbury and Burrow 2006). Therefore, they are forced to
make decisions that are not optimal but only satisfactory according to a pre-set aspiration level. In contrast, expert
designers prefer the breadth first, depth next strategy (Akn 2001). As a result, multiple alternatives help reveal new
directions for further exploration that the designer wouldnt have thought of otherwise. Design teams need dynamic
rule-driven systems that help them set up and manage design generation processes with the right balance of breadth
and depth strategies to best address the multidisciplinary requirements.
2.2 Process modelling
Design theory helps us understand the general process of design; however it does not help us determine how
specifically to represent and measure design processes. Such understanding can help quantify and compare the
performance of existing and proposed processes, as well as provide the tools that help organizations adopt the
proposed processes. A widely accepted implementation method is process modeling. Multiple process models for
AEC have been proposed (Froese 1996). Among other significant process models are IDEF
0
(Integrated Definition
Methods) used to model decisions, actions, and activities of an organization or system, the Narratives (Haymaker et.
al. 2004) that provide a means to model information and the sources, nature, and status of the dependencies between
information, and Value Stream Mapping (Tapping & Shuket 2002) used to illustrate and analyze the flow of actors,
activities, and information that produce value in a given process in order to assist in process re-engineering. Despite
the wealth of existing process modeling methods, an important need specific to this research is not adequately met
a representation formalism for communicating the structure of parametric models and their multiple levels of
multidisciplinary information dependencies.
2.3 Requirements engineering
Poor definition or misunderstandings of requirements are major causes of system failure in software engineering
(Rolland & Salinesi 2005), mechanical engineering (Hsu & Woon 1998), and in AEC (Kiviniemi et. al. 2004).
Systematic methods to screen and prioritize among design requirements have been proposed (i.e. Quality Function
Deployment (Takai & Ishii 2006, Leary & Burvill 2007), PREMISS (Kiviniemi et. al. 2004), MACDADI
(Haymaker & Chachere 2007). While some methods help designers translate requirements into feasible design
80
9
th

options (i.e. Chen & Pai 2005), no systematic method exists for reliably generating parametric design spaces from a
multidisciplinary requirements model. Requirements Engineering (RE) can help formalize such a process. RE is
used as a means to overcome the drawback of traditional software development methods, in which the developed
systems are often technically good but unable to appropriately respond to the users needs (Rolland & Salinesi
2005). RE helps determine the features the system needs to satisfy, and how the system is to be constructed (Ross
and Schoman 1977). Reasoning with requirements can also help resolve conflicts among stakeholders. To develop
better solutions designers need to understand how requirements relate to each other as well as to other elements in
the requirements model. AND/OR graphs are used to capture goal refinement links (Lamsweerde 2000). While RE
provides an actionable method to help designers translate requirements into better solutions, it does not provide a
specific means to translate design requirements into options generated with parametric methods a task that this
research proposes to address.
2.4 Parametric modelling
Parametric computer-aided design (CAD) is a design methodology used to create design spaces and manage
geometric dependencies within a model. The concept of features not present in other CAD systems, encapsulates
generic shapes or characteristics of a product with which designers can associate certain attributes and knowledge
useful for reasoning about that product (Shah & Mntyl 1995). Using parametric CAD tools designers can create an
infinite number of objects, geometric manifestations of a previously articulated schema of variable dimensional,
relational or operative dependencies (Koralevic 2003). However, designing with multiple constraints without an
efficient constraints management system is a daunting task. An example of a constraint management methodology
are the design sheets, in which design models are represented as constraints between variables in the form of
nonlinear algebraic equations organized into bipartite graphs and constraint networks (Reddy et al 1996). Using only
design sheets to define high-rise parametric models would be challenging given the overwhelming number of
constraints that need to be described at the schema level and the inability to visualize geometry. As a result, in
parametric systems, Geometric Constraint Programming (GCP) is used to graphically impose geometric constraints
to solve the relevant nonlinear equations without the user explicitly formulating them (Kinzel et al 2007). This
research, however, identified the lack of a formal method to determine constraints and parameters for constructing
parametric models with sufficient flexibility to respond to a set of multidisciplinary requirements.
2.5 High-rise design methods
This point of departure helps categorize the types of high-rises and elicit a list of design constraints, criteria and
performance metrics that each category entails. In prior work we have reviewed many specific goals and methods
for high-rise design found in literature (Gane & Haymaker, 2008). Understanding and translating these requirements
into quantifiable architectural and energy performance goals and constraint is important in the process of building
parametric models.
3. RESEACH QUESTIONS
Our research proposes to answer the following questions:
What is a method to generate conceptual parametric models of high-rise design spaces that respond both to
multidisciplinary performance requirements?
What is a method to synthesize these requirements models, parametric models, performance analysis models,
and decision making models into an effective and efficient methodology for high-rise design practice?
4. RESEACH METHOD DESCRIPTION
In Figure 1 illustrates the updated conceptual design process called Design Scenarios that we designed to address
these questions. Such a process starts with a clear set of architectural and engineering performance requirements that
help establish the design space. The identified requirements guide the generation of design alternatives within the
established design space. Alternatives are then formally analyzed in discipline specific tools. A bi-directional
relationship between alternatives and analyses supports a recurring refinement process. The multi-attribute
performance of each alternative helps determine its value and establish a formal decision making process. Analyzed
options are then correlated with the requirements to determine their value and choose best option.
The goal of this research is to: help design teams formally identify architectural and engineering performance
requirements and translate them into parameters for developing design alternatives with parametric CAD (link
between boxes 4.1 & 4.2, 4.2 & 4.3, Fig. 1); help determine the value of each analyzed option in relation to specific
requirements (link between boxes 4.1 & 4.4). Parallel research is being conducted at CIFE, Stanford to automate the
81
9
th

engineering performance (i.e. energy, daylight, thermal comfort) analyses generated with parametric CAD (link
between Model based analyses & box 4.4), as well as determine importance of design parameters in relation to
specific requirements (link between Model based analyses & box 4.3).

FIG. 1: Proposed conceptual design process.
Following is a description of the models comprising the Design Scenarios methodology.
4.1 Requirements Model (RM)
The augmented conceptual design process starts with building a RM. Unlike current practice, in which the architect
unilaterally makes most of the early design decisions based on a set of loosely defined requirements, building a RM
will require in addition to stakeholders the participation of all design disciplines (architect, structural, and
mechanical engineers). The RM is hosted online to facilitate remote definition of a comprehensive set of high-level
project requirements by the project users (stakeholders and design team), who prioritize them according to their
level of importance. Each participant has to distribute 100 percentage points to each identified goal, which
represents their preference. Inputs range from stakeholder defined requirements (i.e. architectural brief, budget,
building efficiency) to those established by the multidisciplinary design team (i.e. preferred design language,
daylight factor, energy comfort). The output of this model is a stacked column chart distinguishing the stakeholders
and design teams priorities and constraints. Determining a comprehensive set of multidisciplinary requirements that
helps eliminate the non productive ambiguity in current early decision making practice is the major benefit of
building a RM. The RM also provides the formal value function for evaluating design options.
4.2 Design Scenario Model (DS)
Building a DS will help the design team map requirements to design strategies (i.e. determining means of achieving
requirements). The goals and constraints from the RM are grouped and serve as the initial inputs into the Design
Scenario environment. A DS consists of five levels of hierarchically built information (1) the RM-established
high-level requirements; (2) Action items; (3) Strategies; (4) Parameters; (5) Parametric constraints.
Requirements - Building on concepts defined in the RM, the DS further categorizes and defines the
requirements in terms such as : Quantifiable (i.e. maximize use of daylight in 50% of interior space, provide
50,000 sq m of usable area) and Non-Quantifiable (i.e. use a specific design language). Requirements are
further decomposed into: Goals (objectives i.e. design within $50m budget) and Constraints (requirements
whose satisfaction is compulsory i.e. 50,000 m
2
usable area).
Actions items - Discipline-specific design team leaders decompose each relevant high-level requirement into
Action Items. These are determined by asking the HOW question for each requirement. Building on concepts
from Artificial Intelligence (Lamsweerde, 2001), the DS describes relationships between Action Items through
AND/OR links. For example, to provide daylight in 50% of the building interior designers need to: (a) control
the building orientation; AND (b) control the lease span; OR (c) introduce shading fins; OR (d) introduce light
shelves; AND (e) control window configuration; AND (e) control glass type, etc. All Action Items with AND
links are required to satisfy the original requirement, whereas an OR link illustrates a choice of action;
Strategies In case Action items cannot be directly translated into geometric or material parameters (i.e.
choose window configuration), design team leaders further decompose these into Strategies (i.e. butt glazed,
expressed mullions, unitized panel, etc).
Parameters - Action Items or Strategies are decomposed into geometric and/or material parameters, the value
of which will determine the designs performance in relation to a specific requirement. For example, to
introduce shading fins a designer must create a depth parameter of length type AND an inclination parameter
of angle type after anticipating the need to adjust the shading fins geometry in response to the provide daylight
82
9
th

in 50% of interior quantifiable goal. Parametrically controlling the depth and inclination of shading fins offers
efficient means to refine the geometry after a formal daylight analysis is performed.
Parametric constraints When identified parameters are required to be within specific ranges, these
parameters are decomposed into Parametric constraints (i.e. No. Floors parameter decomposed into Lower limit
30 floors; Upper limit 50 floors parametric constraints).

The DS will also allow design teams to model relationships between the above concepts. The explicit definition of
parameters will help determine potential interdependencies. For example, the shading fins depth input parameter
will impact the shading fin area output parameter, which in turn will determine the shading fin cost parameter used
in calculating the overall design cost. Potential conflicts among requirements may not become apparent just by
developing a Requirements Model, but can be identified when Action Items and Parameters are determined and
related. For example, introduce shading fins Action Item can potentially conflict with design within $50m budget
quantifiable goal given the additional cost of external fins. A DS will explicitly show such conflicts and help the
design team mediate an updated set of requirements with the stakeholders, thus avoiding costly design revisions
common in current practice. Knowledge of these dependencies will guide the CAD specialists (i.e. parametric
modelers) in creating a model that is optimally constructed to address the identified requirements. The DS model
output is a bipartite graph whose orientation, level of detail, and format are user determined.
4.3 Parametric Process Model (PPM)
PPMs help the design team illustrate and manage the logical construct and technical implementation of a DS in a
parametric CAD model. CAD specialists build PPMs. Input, output and constrained parameters and the
relationships/dependencies established in the DS serve as the initial inputs. A PPM also consists of Components
made of geometric and construction elements, PowerComponents made of generic components grouped and
intended to be used in unique contexts, Geometric Constraints used to establish relationships among geometric
elements and parameters, Information Dependencies (i.e. Component A dependent on Component B or Component
A dependent on input parameter(s), etc). A PPM is a formal roadmap to building a parametric model. Parameters
and relationships established in the PPM are used as inputs to automate their generation in a chosen parametric
modeler (i.e. Digital Project, Generative Components). The output of the process is a bipartite graph whose
orientation and format is user determined and the beginning of a parametric model that requires the CAD specialists
to build the components described in the PPM graph and link the automatically generated parameters.
4.4 Options Analysis Model (OAM)
OAMs help the design team evaluate how each option generated from a parametric model and analyzed in discipline
specific tools (i.e. daylight in Radiance) ranks in relation to the high level requirements identified in the RM. Scores
measured in percentage points are assigned to each option based on low and high benchmarks (i.e. high benchmark
minimize cost to $80mln, low benchmark minimize to $100mln). If an option achieves a goal, it receives 100%
score. If it exceeds it (i.e. $70mln, it receives the percentage scored above the high benchmark 112.5%, etc). This
allows design teams to determine the impact of each options performance against the RM goals. Goals, however,
ranked in terms of their importance to each discipline are also measured in percentage points. To determine the final
value of each option, the impact score for each goal is multiplied with the appropriate goal importance score and
summed into a final value function score. The outputs of OAM are spider diagrams and column graphs. An OAM
offers design teams a formal unifying structure and communication tool for describing and managing the
quantitative and qualitative analyses of options.
5. CASE STUDY HIGH-RISE IN SAN FRANCISCO, CA
We are developing a web-based Design Scenarios software platform to significantly improve the DS modelling
process by partially automating the generation of each consecutive model and feed the generated parameters into a
parametric modeller. Prospective validation of DS in practice is expected in 2010-2011. Our research method
(Hartmann et. al. 2008) involves using an embedded researcher who will spend approximately one month in an AE
firm. DS will be introduced in several training sessions. Studio members will have prior experience of using
parametric modelling on several projects. DS will be used by the trained team on several case studies (office
towers), in which the conceptual design process lasts about three weeks. Our goal is to improve DS through iterative
implementation by testing it against the following metrics: (1) goal definition clarity; (2) concept design duration;
(3) No. generated options; (4) team size / composition; (5) total man hours per discipline; (6) time per task; and (7)
explicit analyses performed. We describe our validation of the Design Scenarios using retrospective data and a
83
9
th

hypothetical high-rise project in San Francisco. The illustrated models are abbreviations of larger models for the
purpose of this paper.
5.1 Requirements Model (RM)
The project commenced with a meeting between the stakeholders (developer team) and the senior design team
members (architect, mechanical and structural engineers). The designers objective was to elicit a comprehensive list
of project requirements and help the developers determine target metrics for those requirements that traditionally fall
outside of their domain of expertise (i.e. energy efficiency). The stakeholder-defined architectural brief was used to
establish the initial requirements, most of which were determined to be constraints (i.e. area requirement, height
limitation). Several quantifiable and non-quantifiable goals were also identified (i.e. maximizing building efficiency
to 85%, minimizing construction cost based on the project budget of $80m, design to be widely recognized in San
Francisco). The benefit of this new meeting format became apparent in the subsequent discussion, when the senior
mechanical engineer (normally not present in the first few meetings) suggested several additional goals for
improving the buildings energy efficiency and supported by target metrics (i.e. maximize the use of daylight to
500lux optimal for office spaces; maximize thermal comfort to a range between 22-26
0
C; minimize energy
consumption to 600MJ/m
2
/year (~1000MJ/m
2
/year is the current average). Once the list of goals was accepted by all
parties, every participant individually ranked each goal according to his/her preference, measured in percentage
points. This helped determine the most important goals and the weighting preferences for each discipline. For
example, the developer team, the architect and the structural engineer saw the recognizable design as their leading
requirement where as the mechanical engineer gave more weight to daylight, thermal comfort and energy
consumption goals. The outcome of the meeting was a good understanding of what the project requirements were
and their level of importance to each discipline. These were formally represented in the Constraints and Goals
models (Fig. 2).
FIG. 2 a) Project constraints whose satisfaction was mandatory; b), c) Ranked project goals - participants had to
distribute a percentage of preference (totalling 100%) to each identified goal
5.2 Design Scenario Model (DS)
Back in the studio the senior design team organized another meeting, in which the high-level requirements from the
RM served as the starting point for building the DS. The meeting lasted one day, during which the constraints and
goals were first grouped and decomposed into Action Items (AI). The process was collaborative given that some
constraints and goals resulted in multiple AIs suggested by one or several disciplines. For example, two AIs were
proposed for Recognizable design goal - creating an aesthetically unprecedented design (architect) and a structurally
unprecedented design in San Francisco (structural engineer). These AIs were acknowledged as vital to the project
given the importance level of the parent goal to the developer and the design teams. Both AIs were assigned an
AND Link and therefore required to be implemented in the final design. In case of multiple OR Links, one option
must be selected. Similarly, the mechanical engineer developed AIs for his relevant goals. For example, for the
maximize daylighting goal he proposed three AIs with appropriate goal refinement links (control building orientation
84
9
th

FIG. 3 Case study project Design Scenario Model (DS)

85
9
th

AND Link; introduce lightshelves OR Link; choose glass type according to orientation OR Link). The
architect proposed additional three for the same goal (choose window configuration AND Link; control the lease
span AND Link; introduce central atrium OR Link). The architect assigned an OR Link to the central atrium AI
because of his knowledge of how important minimizing construction cost and maximizing efficiency were to the
developer. This understanding helped identify a potential goal conflict. An atrium in a project constrained by height
and site setback made it impossible to achieve the efficiency and potentially the construction cost goals. Design
Scenarios, however, is built as a recursive process. Therefore, the atrium option was kept in the design scenario in
case the preferences for high-level goals changed.
Some AIs required further decomposition into Strategies. For example, after evaluating the site and its context, the
architect suggested three possible strategies for the aesthetically unprecedented design AI round OR rectilinear
footprint, AND mainly glass exterior. Once all AIs and strategies were finished, each discipline proceeded to
decomposing these into input and output parameters. For example, for the Tower height range constraint the
architect proposed one AI determine No. floors, which was decomposed into Total No. floors AND Floor height
input parameters. When constraints were specified, parameters were further decomposed into Parametric constraints
(i.e. Total No floors -> Lower limit 30, Upper limit 50). When appropriate, parameter interdependencies were
specified for determining output parameters (i.e. Floor height will determine the Window height output parameter).
Without formal knowledge of parametric modelling, the senior design team was able to describe Action Items in
terms of parameters and parameter interdependencies. The abbreviated DS model is illustrated in Fig. 3.
5.3 Parametric Process Model (PPM)
Fig. 4 illustrates the case study PPM. Each component that has a visual preview is numbered. The input variables
and geometric constraints for each component are lettered and their location is shown in the component preview.
The completed DS model served as a starting point for parametric CAD specialists to build the PPM. The AIs
helped determine the goals and constraints that affected the initial decisions of how to build a parametric model.
Input and output parameters were correlated with the appropriate AIs before building PPM components. For
example, the senior architect specified two geometric possibilities for the aesthetically unprecedented design AI -
curved or rectilinear footprint, which required the parametric model to support changes in geometric topology.
Therefore, the Ground footprint (1) (component (1), Fig. 4) was composed of BSpline of order 2 that supported such
transformation. Being a linear BSpline it can either be a line or a curve depending on how it is geometrically
constrained to other geometric elements. The choice of geometry helped determine the geometric constraints
controlling the component and assign the appropriate parameters established in the DS. For example, building side
length (B) input parameter controls the length between the BSpline endpoints, which use a concentric (E) and
tangency (F) constraint to a skeleton of construction (dashed) lines. Tangentially constraining the BSpline endpoints
to the construction lines will change the geometrys topology. The construction lines endpoints are coincidentally
(C) constrained to establish a pin connection. The lines use a perpendicular constraint (D) to avoid arbitrary
rotation. In response to control building orientation AI the rotation angle (A) parameter is introduced by
constraining the angle between the construction line end point and the chosen axis of the user coordinate system.
Having established the ground footprint allows the dependent components to be constructed. For example, Building
core footprint (2) is constrained through length to exterior wall (A) parameter to the ground footprint (1) BSplines,
which established a component dependency. Changes in the footprint impacted the core unless the length to exterior
wall parameter was adjusted to compensate the increase or decrease in the building length parameter. The remaining
components were similarly built.
86
9
th

FIG. 4 Case study project Parametric Process Map (PPM) used for technical implementation of the DS in a
parametric model, managing the model and communicating its construct.
5.4 Options Analysis Model (OAM)
The design team used the resultant parametric model to generate over 1,000 design variations in one week by
operating the values of several key input parameters (i.e. building side length, lease span, floor height, lightshelves
depth, window width). Building an OAM required them to analyze the options to formally understand their
performance. However, most of the options were discarded after not meeting constraints that were defined as output
parameters (i.e. No. floors, building net area, and building efficiency). The options that passed these requirements
were visually analyzed by the senior designers, who chose 20 options (5 shown in this paper) for further formal
analysis outside of the parametric model.
For example, to calculate the daylight performance, mechanical engineers used Autodesk Ecotect. Key parameters
that determined the performance for each option were Lease span, Floor height, Light Shelf depth, and Window
width. The analyses results were compared against the target value of 500lux. However, no formal model based
analysis was performed for thermal comfort and energy consumption. Currently, a widely accepted tool for
performing such analysis is EnergyPlus by the US Department of Energy. Interoperability and model preparation
issues make the use of this tool in early stages of design daunting. As a result, the mechanical engineers evaluated
the thermal comfort and energy consumption performance of the selected options by analyzing a combination of
geometric parameters used to generate each option. For the Minimize energy consumption goal, for example, each
option was evaluated in terms of four key parameters: Lease span, Floor height, Lightshelves depth, Window width.
A deeper lease span, a taller floor height, and a smaller light shelf (used to block direct sunlight along the building
perimeter) meant greater volume to condition with mechanical systems and greater variation in internal temperature,
which made the option less energy efficient.
87
9
th

The OAM required building the impact and value models of each analyzed option. After performing each analysis
on all selected options, the senior designers determined the low benchmark for each discipline specific goal and then
assigned a score representing the performance measured in percentage points. For example, any option that resulted
in an average daylight value of 500lux scored 100% (benchmark determined in the Requirements Model), and 0% if
it had an average of 0lux. The impact score for each option was determined by summing the scores of each goal for
that option (i.e. Option 1 = 515 - Fig. 5a). However, as determined in the RM, goals were ranked according to their
level of importance to the stakeholders and the design team (i.e. Recognizable design = 115 points Fig. 2b). The
OAM required determining the option total value score before choosing a winning design. The sum of each goal in
the RM was translated into a percentage of importance from the total of 400 points that the four participants had to
distribute (i.e. Recognizable design goal scored 115 points or 29% overall importance Fig. 2c). Multiplying each
goals preference score to the impact score determined the goal value of each option (i.e. Option 1, Recognizable
design 29% x 70 = 20). Summing the value of all goals resulted in the overall value per option (i.e. Option 1 = 86 -
Fig. 5b, 5c) Fig. 5d illustrates the parameter values determining the 5 selected options and total value scores.

a) b)
c) d)
FIG. 5 a), c) Case study OAM impact model; b), d) value model used for final decision making option 1 scored
the highest and was considered the winning design.
6. CONCLUSIONS
In previous work (Gane & Haymaker 2008) we have benchmarked current conceptual high-rise design processes in
terms of the metrics listed in Fig. 6. This paper presented a new collaborative design methodology called Design
Scenarios and illustrates a case study of applying DS on a high-rise project. We compare the metrics of current
practice with anticipated metrics from the case study. We wish investigate how a high degree of goal definition
clarity can help a multidisciplinary design team build parametric models and explore and analyze a much larger
segment of the design space in less time. Unfortunatley the case study method of implementation of DS willmake it
difficult to claim generality in our findings.the Design Scenarios methodology and defined metrics can guide
research and development efforts to improve these measurements, and can serve as a benchmark for comparing new
design methods, tools, and processes.
88
9
th

FIG. 6 Comparison between metrics describing current conceptual design performance and the case study results.
Many more design options can be generated and analyzed with Design Scenarios in less time.
7. REFERENCES

Akn, . (2001). Variants of design cognition. Design Knowing and Learning: Cognition in Design Education.
Eastman, C., Newstetter, W., & McCracken, M., Eds., pp. 105124. New York: Elsevier.
Chen, D., Pai, W. (2005), A Methodology for Conceptual Design of Mechanisms by Parsing Design
Specifications. Journal of Mechanical Engineering, Vol. 127, Issue 6, pp. 1039-1045.
Ellis, P., Griffith, B., Long, N., Torcellini, P., Crawley. D., Automated Multivariate Optimization Tool for Energy
Analysis, Proceedings of SimBuild 2006, MIT, Cambridge, Massachusetts. IBPSA-USA.
Froese, T. (1996). Models of Construction Process Information. Journal of Computing in Civil
Engineering. pp. 183-193.
Gane, V., Haymaker, J., (2007). Conceptual Design of High-rises with Parameteric Methods. Predicting the
Future, 25
th
eCAADe Conference Proceedings, ISBN 978-0-9541183-6-5 Frankfurt, Germany, pp 293-301.
Gane, V., Haymaker, J., (2008). Benchmarking Current Conceptual High-rise Design Processes. Submitted to
ASCE Journal of Architectural Engineering.
Haymaker, J., Chachere, J., (2007). Coordinating goals, preferences, options, and analyses for the Stanford Living
Laboratory feasibility study. Conference paper.
Hsu, W., Woon, I. M. 1998. Current Research in the Conceptual Design of Mechanical Products. Computer-Aided
Design, 30, 1998, pp 377-389.
Jin, Y., Danesh, M. (2007). Value Aggregation for Collaborative Design Decision Making.
Keeney R., von Winterfeldt, D. (2007). Practical Value Models. W. Edwards, R. Miles, and D. von Winterfeldt
(eds.) Advances in Decision Analysis New York: Cambridge
Kinzel, E., Schmiedeler, J., Pennock, G. (2007). Function Generation With Finitely Separated Precision Point
Using Geometric Constraint Programming. Journal of Mechanical Engineering, Vol. 129, Issue 11, pp.
1185-1191.
Kiviniemi, A., Fischer, M., Bazjanac, V., Paulson, B. (2004). PREMISS - Requirements Management Interface to
Building Product Models: Problem Definition and Research Issues. CIFE working paper #92
Kolarevic, B. (ed), (2003). Architecture in the Digital Age: Design and Manufacturing. Taylor & Francis.
Kelley, D. Design Thinking. Accessed at http://www.extrememediastudies.org/extreme_media/
1_navigating/pdf/navigating_design_thinking.pdf
Krishnamurti, R. (2006). Explicit design space? Artificial Intelligence for Engineering Design, Analysis and
Manufacturing. 20, 95-103.
Lamsweerde A (2000). Requirements engineering in the year 2000: A research perspective. Proceedings of 22nd
International Conference on Software Engineering, (ICSE2000): Limerick, Ireland, ACM Press, pp. 519.
Leary, M., Burvill, C. (2007). Journal of Mechanical Engineering, Vol. 129, Issue 7, pp. 701-709.
Lewis, K. E., W. Chen, and L. Schmidt (eds.) (2007). Decision Making in Engineering Design. New York: ASME
Press
Reddy, S., Fertig, K., Smith, D. (1996). Constraint Management Methodology for Conceptual Design Tradeoff
Studies. Proceedings of the 1996 ASME Design Engineering Technical Conferences and Computers in
Engineering Conference. Aug. 18-22, 1996. Irvine, California.
Rolland, C., Salinesi, C. (2005). Modeling Goals and Reasoning with Them. Engineering and Managing Software
Requirements.
89
9
th

Ross, D., Schoman, E. (1977). Structured Analysis for Requirements Definition. IEEE Transactions on Software
Engineering, Vol. SE-3, No. 1.
Simon, A.H. (1969), The sciences of the artificial, MIT Press Cambridge MA. USA.
Shah, J., Mntyl, M. (1995). Parametric and Feature-Based CAD/CAM: Concepts, Techniques, and
Applications. Wiley, John & Sons, Inc.
Takai, S., Ishii, K. (2006). Integrating Target Costing Into Perception-Based Concept Evaluation of Complex and
Large-Scale Systems Using Simultaneous Decomposed QFD. Journal of Mechanical Engineering, V 128, Issue 6,
pp. 1186-1196.
Tapping, D., Shuker, T. (2002). Value Stream Management. Productivity Press.
Woodbury, R., Burrow, A., (2006). Whither Design Space? Artificial Intelligence for Engineering
Design, Analysis and Manufacturing, 20, 63-82

90
9
th

AN EXPERIMENTAL SYSTEM FOR NATURAL COLLOCATED AND
REMOTE COLLABORATION

Jian Li,
Department of Civil Engineering, Eastern China Jiaotong University, China.

Jingyu Chen,
Department of Communication, Jiangxi Province, China
ABSTRACT: The phenomenon of convergence between computational and communication technology has created
an absolute breakthrough in the way human interact with each other and how they interact with the technology itself
in particular. In its further development, the convergence has been enhanced to support task-based interaction.
Thorough research on Computer-Supported Cooperative Work (CSCW) and development of groupware are always
complimentary and challenging each other. This abstract proposes an experimental system, a mixture between
electronic peripherals and Human-Computer Interface (HCI) design that allows participants to dynamically interact,
collaborate and communicate with each other, as well as with the system itself. Every user will be projected in real
size at all other participants workspace all of their body gestures, movements, orientation and verbal
communication can be observed by all other participants and being recognized by the system at the same time. This
proposed system will allow users to collaborate on specific works in effective, yet natural workflow. In the system,
awareness in groupware environment is highly maintained by utilizing various technologies that allow critical
awareness information being passed on to all participants. The awareness clues are generated by visual and audio
information. The system aims to provide multi-device collaboration capabilities through the development of an
online collaboration workspace. The focus of this research is to investigate all possible body gestures that can be
used to interact with the collaborative workspace, including the technique used to identify specific body movements
and relate it to the task inside the workspace. On top of that, the research will discuss about the importance for the
system to recognize orientation of the user.
KEYWORDS: Computer-supported cooperative work, design collaboration.
1. INTRODUCTION
There is a huge challenge in developing a groupware system that will further enhance the effectiveness of
productivity of the users (Regenbrecht et al. 2000; Davidson et al. 1996; Schnabel and Kvan 2002). Interactions in
the shared workspace usually limited and unnatural, being limited by inadequate awareness information can be
provided by the groupware system. In fact, awareness is very important knowledge that sustains collaborative works
(1). If collaborative system is not carefully designed according to Kvan (2000) participants might think they are
collaborating, but in fact they are actually cooperating or even compromising because of the system cannot provides
users with the ingredients of successful collaborative works (2). The direction of this research is not about to create a
new type of groupware system, rather than to synthesize the combination of various possible technologies and
emerging knowledge that will allow users to interact naturally with the system and with the other users using set of
gestures and graphical user interface regardless of location and task.

CONACT SYSTEM will allow users to interact with digital and physical objects in the workspace synchronously,
thus creates an unlimited interactivity between users and those objects. In broader scope, it combines technology in
computational system, communication, display and visualization, multi-modal input, image processing and Human-
Computer Interface (HCI). The synthesis of these peripherals will create innovative interactivity between users and
the virtual collaborative environments on elevated degree. By using CONACT SYSTEM, participants not only able
to see each other in full dimension, but also able to see the entire workspace, every objects in it and everything
happened in the workspace simultaneously. The system will recognize body gesture the main input and interaction
method in CONACT SYSTEM. By then, participants provided with more awareness clues and natural interactions.
Therefore, it promotes communication that is more effective, enhances the entire workflow and interactivity during
the collaboration process. The focus of this research is to outline all possible body gestures that can be used to
interact with the collaborative workspace, including the technique used to identify specific body movements and
91
9
th

relate it to the task inside the workspace. On top of that, the research will discuss about the importance for the
system to recognize orientation of the user.
2. OBJECTIVE, ISSUES, AND SAMPLE CASE
According to research conducted by Gutwin and Greenberg (2002), situational awareness of one another is relatively
easy to maintain and the participants can collaborate naturally in spontaneous and effortless. In contrast, awareness
in groupware environment is much harder to maintain compared to face-to-face communication (1). The way that
users operate the system and engaged with other participants are, also important elements that help promote natural
interaction in collaborative environment (3). In CONACT SYSTEM, awareness in groupware environment is highly
maintained by utilizing various technologies that allow critical awareness information being passed on to all
participants. The awareness clues are generated by visual and audio information. Visual information is consists of
the real-size projection of all participants, including their facial expression, body movements and orientation relative
to the workspace. Audio information is consists of all verbal conversation and all sounds originating from any
objects in the workspace. Additionally, intuitive interface is also an important feature in the operation of CONACT
SYSTEM.

CONACT SYSTEM incorporates high-resolution large screen display and vertical display with touch recognition. It
deeply relies on motion tracking sensor and cameras to both capture and recognize the body movement and gestures.
It also combines several microphones to capture participants and environment sound. These devices will capture all
necessary awareness information for all participants. At the central of CONACT SYSTEM, a main control unit
controls the whole system. CONACT SYSTEM can be used as stand-alone collaboration device, or connected with
other computer to extend its functionality. Therefore, the system should be highly compatible with various software
applications, operating systems and hardware platform. CONACT SYSTEM has to be connected to the Internet by
high speed broadband connection in order to supports its data-intensive operational.

This research project explains the importance of utilizing all of those technological devices to deliver adequate
awareness information to the participants and to interact naturally with the workspace through verbal commands and
body gestures. Therefore, participants may be engaged naturally in virtual collaborative environment effortless and
spontaneously.

2.1 Issues
Most of groupware applications cannot handle coordination, cooperation and communication at the same level (3).
Some of them only cover one or two aspects. For example, video conferencing systems are mostly only promotes
communication aspects, while lacking of coordination and cooperation among participants. In different case, online
document editor only assists coordination and cooperation while lacking communication support among
participants.

Nevertheless, the way of users engage with the system and every object in the workspace is also a defining factor of
natural interaction. In face-to-face collaboration, there is no boundary between participants and the environment.
However, in computer-mediated collaborative environment, participants are separated by the system itself and they
cannot interact like in face-to-face collaboration process. In order to create a more sophisticated groupware system,
coordination, cooperation and communication must exist simultaneously at maximum level. This means, participants
can have a good coordination in doing collaborative task, cooperate with others in completing specific task, as well
as communicate with other participants seamlessly. In addition, the system should not restrain participants from
doing collaborative tasks and interacting with every object in the workspace. Therefore, body gesture recognition
will allow users not to deal with complicated operational method of the device, leaving them to collaborate naturally
with other participants. Apparently, most peripherals needed to construct such system are available on the market.
Therefore, it can be assumed that the limitation of building CONACT SYSTEM is rather on arranging and engineer
those peripherals to achieve the qualities mentioned in prior discussion.

2.2 Sample case
It is imaginable how hard it is to collaborate remotely on large-scales, graphical-intensive task such as architectural
planning, product design, and 3D animation. All of these tasks require extensive graphical information, as well as
high level of coordination and communication between people involved and cooperation towards the tasks. In such
92
9
th

condition, complete view of the workspace and other participants, as well as naturalistic interaction with the system
will benefit all participants in the work process. The system offers a close-to-reality experience in a virtual platform.
In other way, the system might be quite redundant in supporting less complex collaborative tasks and
communication, such as application sharing and video conferencing. However, the fundamental functions possessed
by the system will benefits participants, especially regarding the gestural interaction with the system itself which
arguably the future of human-computer interaction method.

3. SPECIFICATION
3.1 Technology involved
In any CONACT SYSTEM set, there is vertical large-screen display, horizontal display with touch-recognition,
video camera to capture the image of participants, motion sensor and image tracker for gesture recognition,
microphones, speaker system and also controller unit which is the brain of CONACT SYSTEM. Figure 1 depicts
the entire system architecture and setup.

Figure 1. The system architecture

93
9
th

3.1.1 Vertical large screen display
The vertical large-screen display is where the images of other participants are displayed. It can utilize either large
panel (> 70 inch) high-resolution LCD display, or LCD projectors. All display devices must have high contrast ratio
and brightness to withstand operational under bright light.

3.1.2 Horizontal (table top) display with touch recognition
Tabletop display is the main place for interaction using CONACT SYSTEM. It also supports touch recognition.
Display resolution is also important the higher the resolution, the bigger the workspace area that can be accessible
by users. It is also preferable to use high contrast-ratio and brightness display. In addition, operational temperature
of the device must not disturb the users, especially for long period of use.

3.1.3 Camera and motion tracker
The camera is used to capture live video of the participants and the workspace area. It supports high frame-rate
recording to reduce ghosting effects. Other camera mounted above the tabletop display can be used as document
camera. It can take a picture of objects placed on the tabletop display. Therefore, users are not only able to interact
with digital objects, but also with physical objects. Motion tracker is used to read body movement and orientation,
especially hand gestures. Motion tracker can be coupled with special gloves that can read fingers movements of each
user, in order to enhance the accuracy of the recognition.

3.1.4 Audio system
In CONACT SYSTEM, the speaker system must be able to produce sound relative to the location of the object
appeared on the vertical screen, as well as on the tabletop display. Several microphones are used to capture
participants voice and sounds in the workspace area. There is ambient noise reduction function to help reducing all
unnecessary noises (such as air condition noise, noises from outside of the room, reverbs, echoes, etc.) and therefore,
allows users to enjoy a clear voice communication.

3.1.5 Control unit
The control unit is a standard computer with custom-made software platform that control the whole CONACT
SYSTEM and do all processing necessary for the operational of the system, including the image processing. It
supports multi-display output, multi-channel sound input and output. It also has both wired and wireless
communication interface to connect with other devices, local networks and the Internet. All components of
CONACT SYSTEM including display units, motion trackers, cameras and audio devices are connected to this
control unit.

3.1.6 Ambient light sensor
Light sensor is added to the system, so it will automatically alter the display brightness to adapt with the light
intensity in surrounding area. Thus, users can always have the right display brightness and comfortable view at the
displays.
3.2 Gesture recognition
The interaction of users with CONACT SYSTEM, is primarily depends on gestural recognition of hands, including
finger touches as well as head and eyes orientation. The recognized gestures are classified into five categories:
native function, basic control, media control, object manipulation and object handling. Each category can
complement each other, but not necessarily related. Usages of gestures are not fixated, but allow combination in
different situations. Table 1 listed all the enabled gestures.

In order to simplify the user interface on each end, only the finger orientation can be seen by other participants.
Other gestural exposures and all user interface that triggered by that particular gesture are not necessarily observable
to other participants. However, other participants should be able to see the actions and reactions caused by certain
gestures. For example, finger orientation will always be visible for all participants, but if a participant rotating and
scaling a picture, the effect of rotation and scaling on the picture can be seen by other participants.

94
9
th

Table 1. List of Gestures

3.3 Design plan
CONACT SYSTEM is not intended for portable use and must be installed in a specific room that can accommodate
space required by the minimal projection distance between the projectors and the screen, as well as the appropriate
distance between the screen and the interactive tabletop. For detailed illustration, see Figure 1.

95
9
th

3.4 Operational and usage
As has been previously discussed, user interacts with the CONACT SYSTEM by primarily using gestures and
touchable graphical user interface. The whole system is controlled by main control unit and connected to high-speed
internet connection for a seamless audio and video communication. A moveable touch screen is available for the
control panel interface for the whole system. At the front, a large-scale display is intended to projects the realistic
size of the other participants and there are several cameras in front of every users for the video communication.
Special software in the main control unit will simulate the gazing direction of a user to other particular user, creating
a more natural communication that closely resembles real-life size objects.

CONACT SYSTEM can be used as a standalone device, as an extension for other computer device, as well as
connected to the local computer network. In addition, other computers and compatible mobile devices can be
connected by both wired and wireless connectivity. The software platform (driver) loaded on those devices will
allow file exchange between the external device and CONACT SYSTEM. CONACT SYSTEM is suitable for
collaborative design, architecture and engineering works, and possibly valuable for remote education device, or just
as a state-of-the-art video conferencing device.

3.5 Evaluation Parameter
As CONACT SYSTEM is only a concept product, it is rather hard to evaluate the usability and effectiveness of such
system. However, the most likely approach of this evaluation process is by using real test case for collaboration in
two remote places, involving at least two participants on each place. To achieve this evaluation procedure, the whole
system must be physically built as working prototype form. While doing the evaluation, the participants should be
encouraged to give feedbacks on the easiness of operating the system and how much effort they have to understand
the operational method (hand gesture, graphical user interface, etc.) of CONACT SYSTEM. The hand gesture must
be assessed as well to be easily understandable, even by new users and the graphical user interface must be assessed
for its ergonomic and clarity.

In addition, as the proposed CONACT SYSTEM framework is huge and complicated, there must be many prior tests
on software and hardware reliability to ensure minimal problems in real-life usage. The bottom line of the evaluation
process is how the system giving benefits to the client, compared with the high price tag. If the clients find that the
system could improve their working efficiency on data-massive, graphical intensive collaborative session, then the
system can be assumed as successful.

4. REFERENCES

Davidson, J.N. and Campbell, D.A. (1996). Collaborative Design in Virtual Space - GreenSpace II: A Shared
Environment for Architectural Design Review, Design Computation: Collaboration, Reasoning, Pedagogy.
Proceedings of ACADIA Conference, October 31 - November 2, pp. 165-179.

Gutwin, C. and Greenberg, S. (2002). A Descriptive Framework of Workspace Awareness for Real-Time
Groupware. Computer Supported Cooperative Work, Special Issue on Awareness in CSCW, Kluwer Academic
Press, 11 (3), pp. 411-446.

Kvan, T. (2000). Collaborative Design: What is It?, Automation in Construction, Vol. 9, pp. 409-415.
Regenbrecht, H., Kruijff, E., Donath, D., Seichter, H., and Beetz, J. (2000). VRAM - a Virtual Reality Aided
Modeler. Proceedings of eCAADe2000, Weimar/Germany.

Schnabel, M.A. and Kvan, T. (2002). Design, Communication & Collaboration in Immersive Virtual
Environments, International Journal of Design Computing, Volume 4; April 2002; ISSN 1329-7147.

96
9
th

URBAN WIKI AND VR APPLICATIONS
Wael Abdelhameed, Ph.D.,
University of Bahrain, College of Engineering, Bahrain;
South Valley University, Faculty of Fine Arts at Luxor, Egypt;
wael.abdelhameed@gmail.com
Yoshihiro Kobayashi, Ph.D.,
Arizona State University, School of Architecture and Landscape Architecture;
ykobaya@asu.edu
ABSTRACT: The research paper involves the implementation of Urban Wiki, an online urban design system
employing Wiki concept, allowing the use of an interactive immersive virtual reality system for visualizations with
dynamic agents such as human and vehicular traffic. The VR system is a platform developed by a software company.
The term Urban Wiki is created by the researchers. Urban Wiki aims to creating a networking system of urban
designs, enabling the collaborative work between users around the world through a VR platform. The presented
system framework is created and tested by the researchers from two different locations in the world. The purpose of
the research is to study how the users can share effectively designing a large scale urban project, and how VR
platform helps in building up the VR urban models to facilitate visualizations and designing. An urban project of a
village scale, which was conducted by one of the researchers is used to demonstrate the potentials of Urban Wiki,
presenting its functions and highlighting the possible uses in the urban area. Moreover, using the created models in
the VR platform that enables visualizations with dynamic agents opens various urban paths of designing, decision-
making, sharing, and communication with the stakeholders, decision makers, and planners. Techniques employed in
the design of Urban Wiki can be potentially used to build scalable, easily navigable and extensible models of large-
scale entities. Combining the application of the two systems, Urban Wiki and VR platform, will be designed as an
intuitive simulation tool, helpful in identifying novel approaches for control and visualization in such applications
as urban design, urban plan and Land Use Physical- Plan.
KEYWORDS: Urban Wiki, VR Applications, VR Platform, Collaborative Design.
1. INTRODUCTION
Collaborative design concept enables users to modify the content of a file from different places around the world.
The available modifications at the beginning were exclusive to the text format, and then other formats such as image
were adapted. Yet, the collaboration in the designing process itself especially on the 3D level is not effectively
introduced. Editing or modifying a 3D design file through a computer system that displays the content of this 3D
design file has not been investigated.
Although there is no enough literature in the area of 3D collaborative designing, some researches investigated
similar areas. For example, Yamashita et al. (2006) developed a collaborative design environment which considers
Information and Communication Technology and architectural space, through supporting synchronous design
collaboration in a face-to-face meeting at a local site and also in a continuously connected project-room at
distributing sites (Yamashita et al., 2006). Lan and Chiu (2006) demonstrated a Web 3D-GIS approach to develop
the urban information system. Lan and Chiu proposed that a digital city should be able to not only visualize a large-
scale 3D city model but also integrate useful urban information for potential users retrieval in a web environment
(Lan and Chiu, 2006). Matsumoto, Kiriki, Naka, and Yamaguchi (2006) proposed the collaborative design education
program on the web, and developed the special Design Pinup Board system for running it. The introduced program
focuses on very limited environment; distributed collaboration beginners, asynchronous, first meeting, and plural
teams (Matsumoto, et. al., 2006). Lee (2001) maintained the possibility to create a 3D modeling tool based on the
recognition of labels in freehand sketches, and introduced a symbol-based 3D modeling tool (the SpaceMaker) that
allows designers to make freehand floor-plan drawings to explore the initial concept of spatial layout and allows
users to apply labels to identify different types of space (Lee, 2001).
97
9
th

Offering the opportunity to modify a 3D design through a networking system at different locations around the world
has been effectively introduced only on the commercial level, such as Secondlife and CityEngine. Second Life is a
3D virtual world. Its residents are allowed to build, own and retain the rights to their digital creations. They,
therefore, can buy, sell and trade with other residents. The Marketplace currently supports millions of US dollars in
monthly transactions. Another example related to the same approach is Massively Multiplayer Online Role-Playing
Game (MMORPG) Ultima Online that was created in 1997. CityEngine is 3D modelling software for urban
environments. It enables its users to build their own neighbourhoods, urban areas and cities with certain types of
buildings and houses.
Linking the concept of collaborative design on the 3D level to the famous Wiki concept is one of the concerns of
this research. Urban Wiki is an online urban design system employing the Wiki concept. The research concern,
moreover, is to investigate the use of an interactive immersive virtual reality system during employing the Urban
Wiki.
2. RESEARCH OBJECTIVES
Urban Wiki implements the requirements and objectives that can be summarized in the following:
Sharing an urban plan through/in a file of max scripting;
Using a 3D modeling system. Its transformations and changes of form assembling are through only
three simple buttons (create, edit and delete);
Applying functions of searching (zoom in and out) and modifying (create, edit and delete) through
networking; and
Linking the urban file to a VR platform.
3. WIKI AND URBAN DESIGN
A Wiki is a website that allows visitors to add, remove, edit and change its content, typically without the need for
subscription. A wiki is an effective tool for mass collaborative authoring through this easiness of interaction among
its visitors. Urban Wiki of the same Wiki potentials is investigated by the researchers in another research paper.
Moreover, Wiki concept on the design level, Design Wiki, was previously introduced by the researchers*******.
Design Wiki has a networking 2D/3D visual design map, DesignMap, through which Design Wiki visitors can edit
the existed designs and then save the modified designs in series based on their topological properties.
The main objective of this research is to share effectively the designing process of a large scale urban project, and to
allow visualizing the urban file through a certain VR platform. Therefore, Urban Wiki investigated by this research
is focused on activities of a group of interest during conducting an urban planning project.
3.1 Urban Wiki
The public Web portals that appeared in the mid-1990s, such as Yahoo, Msn, AltaVista, and Excite, have portlets
that provide self-contained boxes for different features like e-mail, news, weather and search. By the late 1990s,
software vendors began to produce pre-packaged enterprise portals, which would be toolkits for enterprises to
quickly develop and deploy their own customized enterprise portal. There are many enterprise portal vendors such
as, Apache Software Foundation (its product name is Jetspeed 2.1), IBM (its product name is WebSphere Portal
Server 6.0.1), and Microsoft Office (its product name is Sharepoint Server 2007).
Urban Wiki is programmed as a main Portlet. Urban Wiki portlet, which is programmed in Java, is a reusable
interface for online applications, and it is running with JetSpeed2 framework provided by Jakarta Project. The
portlet can be run through all kinds of Portal applications. It was tested using a free Enterprise Internet Portal
Framework, JetSpeed2 (by Apache Portal Project site, http://httpd.apache.org/).
3.1.1 Methodology
The user has to prepare two files; an initial space layout file and a file of space property list. The file of Space
Property List has the adjacency list of a design, for example in Figure 1, the adjacency list for 43 is: 8, 22, 40, 39,
30, and by changing the three outlined cells to a new space, e.g. 45, the list will be: 8, 45, 8, 22, 40, 39, 30. The
98
9
th

Space Layout file should have at least one design with an array of 400 (20x20) integer numbers such as (0 0 0 0 1 1
1 . 0 0 0).

FIG. 1: A displayed 2D design with the three buttons of modifying by adding the three outlined cells
Each integer number represents a space property such as 0=Street, 1=Sidewalk, and 2=House, which are defined the
file of space property list, Figure 2. By changing the properties of cells in the portlets grid, new spaces are created.
A space is defined as a set of cells with the same property such as house, front yard, driving way, and garage.

FIG. 2: A design displayed in 2D and 3D with a space list

99
9
th

After creating the design or the urban plan, it can be added to the main urban file of max scripting. Visitors, a group
of interest, who have the access to the main urban project file can add, edit or delete designs of this urban project.
Figure 3 presents a 2D urban plan project that was used to test the application.

FIG. 3: 2D maps, land use and master plan, of the urban plan project, Arabah Sharqeyah Village, Sohag
Governorate, Egypt.

3.1.2 Functions
The functions offered by Urban Wiki were tested by the researchers through making modifications at two different
locations to the urban file. The interactive modifications through the VR allow to visualize the impact of these
modifications and to link them to other factors. The influential factors assessment and their impact in an interactive
VR environment help not only urban designers but also the associated partners and stakeholders.
3.1.3 Abstract outputs
Urban Wiki applies only square grids which would generate straight shapes and forms. Although diagonal lines and
polygonal shapes can be adapted and implemented by Urban Wiki, the research paper employs an abstract style for
the created designs possibly added to the urban models. Whereas the focus on the large scale projects of urban
designing and urban planning is directed to the surrounding environment rather than buildings and their details.
100
9
th

3.2 The Urban Project
On large scale urban projects, many authorities and individuals are involved, for example municipalities, district
councils, stakeholders, decision makers, and planners. A major part of the designing process itself is the effective
communications between the different parties. Facilitating the processes of sharing, communication, decision
making, and visualization is the main advantage of employing the Wiki concept in urban design projects.
The urban project presented by this research is of Arabah Sharqeyah, a Village at Sohag Governorate in the middle
of Egypt. During this urban project, there were several meetings with the inhabitants of the village and the members
of village council in order to share in the decision making processes whenever it is approved by the planning
authorities and the Ministry of Housing and Planning.
The urban model was created by the 3ds max file and displayed in Urban Wiki system, Figure 4. The final output
that has modifications made by the Urban Wiki is imported to the VR program platform in max scripting file format.
The link between Urban Wiki and a VR platform eases many activities and tasks of the urban designing. Of the
previous experience resulted from conducting this urban project, the VR models with the possibility of interaction
offer a more effective way than of provided by the static models.

FIG. 4: The user interface of the Urban Wiki System.

101
9
th

Based on the Space file used in the Urban Wiki, each design can present certain data. This flexibility is important to
adapt different styles of urban fabric of different urban projects. Different models would be built up to visualize and
evaluate the alternative solutions and various influential factors of the urban environment. These changes and
modifications can be visualized through the Urban Wiki system at different locations at the same time through
networking.
4. VR PLATFORM
The final step is to import the main urban project file to a VR platform. The platform used is UC Win/Road, a
software program developed by the Forum8 Company. Its version 3.04 has the function of importing different file
formats such as shp, max script, 3ds max and dwg. Urban Wiki employs the option of max script format. Also, the
latest version, VR studio, has the function to visualize the same VR model through networking at computers of
different locations. Figure 5 shows a screenshot of the VR model of an urban project, displayed in UC Win/Road.

FIG. 5: A screenshot of the VR program showing the village model.

The VR platform, UC Win/Road 3.04, enables visualizations of traffic simulation with dynamic agents such as cars
and pedestrians. This interactive immersive virtual reality system opens various urban of designing, decision-
making, sharing, and communication with the stakeholders, decision makers, and planners. These applications of
Virtual Reality platform are the concern of this research. Various urban project problems can be solved while
different scenarios can be visualized for evaluation and comparisons.
On another research contribution, techniques employed in the design of Urban Wiki can be potentially used to build
scalable, easily navigable and extensible models of large-scale entities. On a village scale or district level, navigation
102
9
th

through the whole project in terms of adding or modifying certain urban areas is highly important. Providing this
function at different locations and visualizing the outputs within these locations at the same time, through Urban
Wiki system, are a major part in the design process of urban projects.
5. SUMMARY AND DISCUSSION
The research concludes to a unique urban system, Urban Wiki, where its objectives, functions and methodology
were illustrated. The contributions and applications of Urban Wiki, through networking and its possible link to a VR
platform can be employed effectively in urban planning as tools of designing, decision-making, sharing, and
communication. VR models are improving the practice of urban environmental planning and design. The visual
display capabilities enable the explanation of the development plans, alternatives or various scenarios to both the
urban project team and the public.
Although the research paper did not concentrate on the urban planning project itself, some modifications were made
in the urban design at different locations countries- through networking in terms of testing and simulating the same
conditions during conducting the project.
There are areas and factors which can be considered crucial urban planning issues that present focuses of future
research papers to be investigated on the urban level.
6. ACKNOWLEDGEMENTS
This research was funded through the research project 4/2008 by the University of Bahrain. We would like to
express our appreciation to the University of Bahrain. Also, our thanks is to Forum8 company for the help they
offered.
7. REFERENCES
Kobayashi Y. and Abdelhameed W. (2008) Implementations and Applications of DesignMap: Case Studies to
Manage an Online Database of Visual Designs International Journal of Architectural Computing, vol. 6 -
no. 3, 243-258.
Lan U-H. and Chiu M-L. (2006) A Web 3D-GIS Approach to Develop the Urban Information System of Virtual
Anping. Proceedings of the 11th Conference on CAADRIA, Kumamoto, Japan, 479-486.
Matsumoto Y., Kiriki M., Naka R. and Shigeyuki Y. (2006) Supporting Process Guidance for Collaborative Design
Learning on the Web: Development of Plan-Do-See cycle based Design Pinup Board. Proceedings of the
11th Conference on CAADRIA, Kumamoto, Japan, pp. 71-80.
Yamashita, S., Yoshitaka M., Yuji M., Ryusuke N., and Shigeyuki Y. (2006) Enhanced and Continuously
Connected Environment for Collaborative Design Proceedings the 24th Conference on eCAADe,
Communicating Space(s), Volos (Greece) 2006, pp. 478-485.
Lee M.-C. (2001) SpaceMaker: A Symbol-based Three-dimensional Computer Modeling Tool for Early Schematic
Development of the Architectural Design. M.Sc. Thesis, Design Machine Group, University of Washington,
Washington, USA.
http:www.secondlife.com/ [15-7-2009].
http:www.cityengine.com/ [15-7-2009].
http:www.forum8.co.jp/english/english0.htm [15-7-2009].

103

AUTOMATION AND INTERACTION

Toward Affective Handsfree Human-machine Interface Approach in
Virtual Environments-based Equipment Operation Training--------------------------107
Iman Mohammad Rezazadeh, Xiangyu Wang, Rui Wang and Mohammad Firoozabadi

Construction Dashboard: An Exploratory Information Visualization Tool
for Multi-system Construction----------------------------------------------------------------117
Cheng-Han Kuo, Meng-Han Tsai, Shih-Chung Kang and Shang-Hsien Hsieh

Computer Gaming Technology and Porosity-----------------------------------------------127
Russell Lowe and Richard Goodwin

Virtual Reality User Interfaces for the Effective Exploration and
Presentation of Archaeological Sites----------------------------------------------------------139
Daniel Keymer, Burkhard Wnsche and Robert Amor

Interactive Construction Documentation----------------------------------------------------149
Antony Pelosi

Case Studies on the Generation of Virtual Environments of
Real World Facilities-----------------------------------------------------------------------------155
Michele Fumarola and Ronald Poelman

Evaluation of 3D City Models Using Automatic Placed Urban Agents----------------165
Gideon Aschwanden, Simon Haegler, Jan Halatsch, Rafal Jeker, Gerhard Schmitt
and Luc van Gool

Integration of As-built and As-designed Models for
3D Positioning Control and 4D Visualization during Construction---------------------177
Xiong Liang, Ming Lu and Jian-Ping Zhang

Augmenting Site Photos with 3D As-built Tunnel Models for
Construction Progress Visualization----------------------------------------------------------187
Ming Fung Siu and Ming Lu

Automatic Generation of Time Location Plan in Road Construction Projects------197
Raj Kapur and Nashwan Dawood

Development of 3D-Simulation Based Genetic Algorithms to
Solve Combinatorial Crew Allocation Problems-------------------------------------------207
Ammar Al-Bazi, Nashwan Dawood and John Dean

Integration of Urban Development and 5D Planning--------------------------------------217
Nashwan Dawood, Claudio Benghi, Thea Lorentzen and Yoann Pencreach
9
th
TOWARD AFFECTIVE HANDSFREE HUMAN-MACHINE INTERFACE

APPROACH IN VIRTUAL ENVIRONMENTS-BASED EUQIPMENT
OPERATION TRAINING

Iman Mohammad Rezazadeh, Ph.D Candidate
School of Biomedical Eng., Science and Research Branch of Islamic Azad University, Tehran, Iran
rezazadeh@srbiau.ac.ir, i.rezazadeh@arch.usyd.edu.au

Xiangyu Wang, Dr.
Design Lab, Faculty of Design, Planning and Architecture, University of Sydney, Sydney, Australia
x.wang@arch.usyd.edu.au

Rui Wang, Master of Philosophy Student
Design Lab, Faculty of Design, Planning and Architecture, University of Sydney, Sydney, Australia
rwan9009@uni.sydney.edu.au

Mohammad Firoozabadi, Prof.
School of Biomedical Eng., Science and Research Branch of Islamic Azad University, Tehran, Iran
Medical Physics Dept. Tarbiat Modares University, Tehran, Iran
pourmir@modares.ac.ir

ABSTRACT: Using Virtual Reality (VR) technology for training is becoming more and more interesting to many
applications ranging from medical to industrial purposes. In the construction arena, some progresses have been
achieved by researchers to design and implement environments for task training using VR technology and its
derivatives such as Augmented and Mixed Reality. However, there are still many shortcomings in this area, which
should be considered. Usefulness and usability of the virtual training environments are two of the most important
factors when designing and implementing them. The usefulness factor can be achieved by designing the virtual training
environments in a way that it can support training phase or could be a substitute for early stage of training in the real
environment, but eliminate the real environment training drawbacks such as high cost, high risk and high difficulty of
repetitive practices. On the other hand, for usability factor, we should suppress the mental and cognitive pressure and
stress over the user whilst he or she is being trained. In the work presented in this paper, we designed a virtual lift
(crane) which can be controlled using commands extracted from facial gestures and is capable of lifting up
load/materials at virtual construction sites. Then we utilized Electroencephalogram (EEG) signals collected from the
users during training to extract their affective measures, which mirror trainees level of comfort. This measurement can
also be used for further tasks such as re-designing the environment and also can be used to stop training until the
trainee reaches the relax state to continue the training phase.
KEYWORDS: Virtual Reality, Construction training, Affective measures, Facial bioelectric signals
1. INTRODUCTION
Construction equipment operators usually operate one or several various types of construction equipment. Virtual
technologies afford new opportunities for effectively training novices with lower cost and fewer hazards. Seidel and
Seidel and Chatelier (1997) have even suggested, for example, that the use of virtual environments (VEs) may be
trainings future. Virtual environments can be especially valuable where training in real-world situations would be
impractical because a real field scenario may be unduly expensive, logistically difficult, dangerous, or too difficult to
control. This approach is envisaged to facilitate progress along what is a steep learning curve and enable effective
rehearsal of future operations in actual construction sites. Construction training research has begun to explore Virtual
Reality as training vehicles. Virtual reality training systems have already provided added benefits to many training
packages. Virtual environments (VEs) embody many of the characteristics of an ideal training medium (Psotka, 1995;
Schroeder, 1995); VEs have already been developed for training of drivers (Mahoney, 1997), firefighters (Bliss,
Tidwell, & Guest, 1997), pilots (Lintern, Roscoe, Koonce, & Segal, 1990), console operators (Regian, Shebilske, &
Monk, 1992), naval officers in ship manoeuvres (Magee, 1997), soldiers in battlefield simulations (Goldberg & Knerr,
1997; Mastaglio & Callahan, 1995), and ground control team for familiarity with the operability of the Hubble Space
Telescope (Loftin, et al., 1997).
107
9
th
Apart from the noted existing VR research in training, recently, it becomes an important research issue in the robotics
field to recognize emotional states such as joy, teasing, fear, sadness, disgust, anger, surprise, and neutral from human
bioelectric signals. A number of studies have been done in this area, ranging from psycho-physiological measures such
as heart rate, EDA pupillometrics or fEMG to speech or video analysis. From different kinds of survey results such as
questionnaires and interviews, it is clear that the success rates are 50-60% in emotional speech recognition and 80-90%
in facial expressions. As we know, physiological indexes are useful to evaluate emotions since they can be measured
physically and objectively and can be easily applied to engineering approaches. One of these approaches which have
been gaining many attentions is affective computing. Affective computing basically means computing that relates to,
arises from, or deliberately influences emotions and it focuses on creating personal computing systems having ability to
sense, recognize and understand human emotions, together with the skills to respond in an intelligent, sensitive and
respectful fanner toward the user and his emotions (Fernandez, 1997; Healey & Picard, 2000; Nasoz, Lisetti, Alvarez, &
Finkelstein, 2003; Picard, 1997).
Ang et al. (2004) stated that facial muscle movements and forehead electromyogram (fEMG) can be corresponding to
certain facial expressions and are the most important visual representation of a person's physical emotional states.
Mahlke and Minge (2006) used emotional states which were extracted from fEMG to discriminate between usable and
usable computerized contexts and they concluded that the frowning activity is significantly higher in the unusable
system condition than in the usable one by pacing two pairs of electrodes on zygomaticus major and corrugators
supercili to detect positive and negative emotional states, respectively. Neimenlehto et al. (2006) studied the effects of
affective interventions using fEMG in a Human-Computer Interaction (HCI) and concluded that the frowning activity
attenuated significantly after the positive interventions than the conditions with no intervention. Kim et al. (2006) used
fEMG Linear Prediction Coefficients (LPCs) by placing a pair of electrode on temporalis muscle and control an electro-
powered wheelchair (EPW) using clenching left, right and both molar teeth and eye blink and classified them using a
Hidden Markov Model (HMM) and achieved 96.5% and 97.1% decimation rate for handicapped and healthy groups,
respectively. Nasoz et al. (2003) have conducted a study to model user emotional state. Three physiological
measurements were used and data was collected from 31 participants (male and female) from student population. This
study used normalized signals instead of statistical features and employed two separate classification method k-nearest-
neighbor (KNN) and Discriminant Function Analysis (DFA). The best results were achieved using DFA to classify 5
emotions with 90% accuracy for fear, 87.5% for sadness, 78.58% for anger, 56.25% for surprise and 50% for
frustration. Healey and Picard (2000) have used 4 physiological signals to detect the intensity of stress in automobile
drivers. Sequential Forward Floating Search (SFFS) was used to recognize patterns of stress whereby the intensity of
stress was recognized with 88.6% accuracy. Fernandez (1997) used two physiological signals to detect frustration in
computer users.
This paper is dedicated essentially to the creation of compelling virtual environments within which human participants
are led to feel somehow present, for purposes of training. More specifically, a novel facial multi-channel bio-electric
signals processing approach was developed to extract affective measures while performing some pre-requested and
sudden tasks. The results of this research could enable better development, implementation, and assessment of virtual
environments for equipment operators. Given the size of the construction industry and other related industries (e.g.,
manufacturing), the results of this research are expected to directly impact workforce and economy. Section 2 presented
the novel facial multi-channel bio-electric signals processing approach to extract affective measures. Section 3
discussed the results from the preliminary study based on the approach.
2. METHOD
The general block diagram of our method to design a bioelectric interface is depicted in Figure 1.

108
9
th

Figure 1 General Block diagram of the proposed system

2.1 Site selection and electrodes placement
Before the placement of any electrode, the selected area should be cleansed from dust, sweat and fat layer to reduce the
effect of motion artefacts. As illustrated in Figure 2, three pairs of rounded pre-gelled Ag/AgCl electrodes were placed
on the volunteer's facial muscles in a differentiation configuration to harness the highest amplitude signals (Firoozabadi
et al., 2008):
One pair on his frontalis muscle: above the eyebrows with 2cm inter-electrodes distance. (Channel 2)
Two pairs placed of left and right temporalis muscles. (Channel 1 and 3)
One ground electrode on the boney part of the left wrist.

2.2 Data acquisition system
The Biopac system (MP100 model and ack100w software version) (Biopac, 2009) was used to acquire bioelectric-
signals. It can collect bioelectric-signals accurately with the selected sampling frequency and store them in its own or
PC memory (1.73 GHz, 2G RAM). The sampling frequency and amplifier gain were selected at 1000 Hz and 5000,
respectively. The low cut-off frequency of the filter was chosen 0.1 Hz to avoid motion artefacts. In addition, a narrow
band-stop filter (48Hz-52Hz) was used to eliminate line noise.

109
9
th

Figure 2- Illustration of the electrodes configuration over frontalis and temporalis facial muscles (Firoozabadi et al.,
2008)

2.3 Data collection protocol
A pilot study was conducted to test the effectiveness and usability of this affective computing-based virtual training
system. Three healthy volunteers have been chosen for this study (male and aged 22, 25, and 29 years old respectively).
Each volunteer rested for 5 minutes prior to recording session. Then he was asked to moderately perform the facial
gestures according to Table 1, 10 times (trails); the data from the 3 channels (3 pairs of facial electrodes) was then
recorded for a period of 2 seconds and started 1 second after gesture generation. There was a 10-second interval
between each trail in order to eliminate the effect of fatigue.

Table 1. Gesture Name and Related movement.
Related Command Gesture Name Gesture Index
No.
Move Forward Smiling 1
Move Right Pulling up right lip corner 2
Move Left Pulling up left lip corner 3
Move Backward Opening mouth (like to say 'a' in 'apple') 4
Lift/Release the load Clenching Molar teeth 5

2.4 Data pre-processing and manipulatory feature extraction
The acquired data from the three channels was passed through a band-passed butterworth filter ranges 30-450Hz which
covered the most significant spectrum related to facial electrical activity. Then the data was divided into non-overlapped
256msec slots and for each slot the root mean square (RMS) value was calculated as a manipulator feature (Ri).
(Formula 1)

R
i
= RMS(EXG
i
) =
EXG
i
2
dt
0
T
T
(1)

110
9
th
Those manipulator features whose values were greater than the threshold, were considered as active features (Ti)
(Formula 2).

T
i
= R
i
3Mean RMS EXG
Quiescent
( ) ( )
+ 3std RMS EXG
Quiescent
( ) ( ) { }
(2)

Finally, the normalized manipulator features were achieved using Formula 3:

S
i
=
T
i
Mean RMS EXG
Quiescent
( ) ( )
(T
i
[ ]
RMS EXG
Quiescent
)
( )
i=1
K
(3)

2.5 Classification
Extracted features need to be classified into distinctive classes for the recognition of the desired gesture. In addition to
inherit variation of bioelectric-signal over time, there are external factors, such as changes in electrodes position,
fatigue, and sweat which may cause changes in a signal pattern over time. A classifier should be able to cope with such
varying patterns optimally, as well as prevent over fitting. Classification should be adequately fast to meet real-time
processing constraints. A suitable classifier has to be efficient in classifying novel patterns; online training can maintain
the stably of classification performance over a long-term operation (Oskoei and Hu, 2007).
The idea of fuzzy clustering is to divide the data space into fuzzy clusters, each representing one specific part of the
system behaviour. Fuzzy c-means is one the fuzzy clustering methods which is a supervised algorithm, because it is
necessary to tell it how many clusters c to look for. If the number of centres is not known before, it is necessary to apply
an unsupervised algorithm. Subtractive fuzzy-means clustering (SFCM) is based on the measurement of the density of
data points in the feature space. The idea is to find regions in the feature space with high density of data points. The
point with the highest number of neighbours is selected as centre for a cluster. The data points within a pre-specified,
fuzzy radius are then removed (subtracted), and the algorithm looks for a new point with the highest number of
neighbours. Subtractive clustering uses data points as the candidates for cluster centers, instead of grid points as in
mountain clustering. This means that the computation is now proportional to the problem size instead of the problem
dimension. Based on the above description, the k-folds algorithm was applied to the training set; where k equals to 10.
The k-1 folds were used to train the classifier and applied to SFCM to derive fuzzy inference system and the rest 1 fold
used to validate it (Moertini et al., 2002; Priyona et al., 2003).
2.6 Affective feature extraction
The Wavelet Packet Entropy (WPE) appears as a measure of the degree of order/disorder of the signal, so it can provide
useful information about the underlying dynamical process associated with the signal.
Assume that wavelet coefficient are given by (4) where S and are the main signal and the corresponding kernel at
shift k and scale j; j=-1,-N; the energy at each resolution level will be the energy of the detail signal and the energy at
each sampled time k will be (5). In consequence, the total energy can be obtained by (6).

C
j
(k) = S,
j,k
(4)

E
j =

k | C
k(k)|
2
[ ]
(5)

E(k)
[ ]
= ( j = N)
(1) | C
j(k)|
2
[ ]
(6)
The relative wavelet packet energy for the resolution j is defined as (7.1 and 7.2):

E
tot
S
2
= E
j
j<0
(7.1)

P
j
=
E
j
E
tot
(7.2)
111
9
th

Now, the Shannon entropy gives a useful criterion for analyzing and comparing probability distribution, providing a
measure of the information of any distribution given by (8).

S
WT
= S
WT
( p) = p
j
j<0
ln( p
j
) (8)

The acquired data from Channel 2 was passed through a 13-21 Hz band-passed filter to gain beta band of brain
electrical activity, which is related to mental activities. Then the filtered data was divided into non-overlapped
1000msec time slots and The Wavelet Packet Entropy (WPE) method was applied the slots to obtain relative entropy
for each pre-defined task known as affective features.
The main reason to choose Channel 2 for acquiring the affective measures is it can capture forehead bioelectric signals
and according to manipulation commands, there was no source of manipulation commands in the forehead.

2.7 Virtual crane and its control

Two different virtual environments have been created. The first one (VE1) was an in-house virtual crane built by
MAYA and it could lift, move and place a virtual load according to manipulation commands as seen in Figure 3.

Figure 3: VE1: virtual crane could be used to place the load on the other side of the wall

Figure 4: VE2: virtual portal crane could be used to place the load in different locations

The second virtual environment (VE2) was also a portal crane from HUMUSOFT s.r.o. and the MathWorks, Inc.(
Figure 4). It could lift the virtual load and be controlled via maipulatory commands from facial gestures to make the
desired trajectory of movement.

3. EXPERIMENTAL RESULTS AND ANALYSIS
Each of the three volunteers were asked to perform data collection protocol. Table 2, shows the achieved result to
discriminate the mentioned facial gestures. It is clear that the classifier has good power to discriminate between facial
gestures. Also, the generating of these gestures are natural and that is why this gesture combination could be used as a
good interface for human machine applications.
112
9
th

Table 2. Average discrimination ratio for mentioned facial gestures for 3 users
Discrimination
Ratio %
Gesture Name Gesture Index No.
90 Smiling 1
85 Pulling up right lip corner 2
96 Pulling up left lip corner 3
100 Opening mouth ( like to say 'a' in 'apple') 4
92 Clenching Molar teeth 5

Then, each human subject was asked to control VE1 and VE2 according to following protocols:
- VE1: he should lift the load and pass it through the wall release it in the other side of the wall
- VE2: he should lift the load, pass it to the opposite corner of the ground and release it.

Both VE1 and VE2 were set to be performed in three different levels of difficulties:
- Level 1(Easy): normal crane movement speed
- Level 2 (Difficult): Fast crane movement speed
- Level3 (Difficult): Slow crane movement speed

Figure 5 shows that there is difference in entropy value when performing Level 1 (blue), level 2 (red) and level 3
(green). That is because the mental and cognitive stress were more higher when performing more difficult task and
Channel 2 could be considered as a good affective channel which can mirror user internal feeling about the protocol.
Also, the average time to complete VE1 and VE2 considering all levels of difficulties are 12.21 sec and 17.68 sec.

Figure 5- Average entropy for Beta band levels of different levels of difficulties for VE1 and VE2. (Blue: Level 1, Red:
Level 2 and Green : Level 3) for case 1 and case 2.

4. CONCLUSION AND FUTURE WORKS
In the work presented in this paper, we developed our previous work on facial multi-channel bio-electric signals
processing approach (Firoozabadi, Oskoei , & Hu, 2008) to extract affective measures while performing some pre-
requested and sudden tasks. In order to test and justify this approach, a virtual environment was created in the presented
study where a virtual carne has been built which can be controlled and manipulated using facial bioelectric signals. A
pilot study was implemented by inviting three human subjects and data was collected. These signals were captured
when facial gestures were generated by users. By using Channel 1 and Channel 3 signals and SFCM, the average
113
9
th
discrimination is 92.6% for the system to discriminate between 5 different facial gestures. The main reason to choose
these facial gestures are their generation source are far enough from Channel 2 and thus Channel 2 can be responsible to
extract affective measure without facial gestures interference. It was also shown that Channel 2 can mirror user
emotional state as the entropy of these Channel rises up when the level of difficulties was increased.
On another note, this training interface can be adapted for the users with disabilities from the user's neck down. Thus, it
can be a window to get to activities of daily life and these people can be more involved in the social activities. In our
futures steps, by using affective measures, the system can automatically adapt itself to the user status to decrease the
mental and cognitive stress while performing the operation.

ACKNOWLEDGEMENTS
We would like to thank Dr Christian Jones from USC-Australia for sharing his expertise on the area of
affective computing and also the help from eagerly volunteer participants is appreciated.

5. REFERENCES:
www.brainfinger.com, Accessed on 5 July 2009
Adams, N. (1996). A Study of The Effectiveness of Using Virtual Reality to Orient Line Workers in a Manufacturing
Environment. DePaul University, Chicago, IL.
Ang L., Belen E., Bernardo R A., Boongaling E. R., Briones G. H. and Coronel J. (2004). Facial Expression
Recognition through Pattern Analysis of Facial Movements Utilizing Electromyogram Sensors. IEEE
TENCON2004, 600-603.
Anon (1994). Immersive VR Tests Best. CyberEdge Journal, 4(6).
Anon (1996). VR in Industrial Training. Virtual Reality, 5(4), 31-33
Biopac (2009). http://www.biopac.com, Accessed on 5 July 2009.
.Bliss, J. P., Tidwell, P. D., & Guest, M. A. (1997). The Effectiveness of Virtual Reality For Administering Spatial
Navigation Training To Firefighters. Presence: Teleoperators and Virtual Environments, 6, 73-86.
Fernandez, R. (1997). Stochastic Modelling of Physiological Signals with Hidden Markov Models: A Step toward
Frustration Detection in Human-Computer Interfaces.
Firoozabadi, S. M. P., Oskoei , M. A., & Hu, H. (2008). A Human-Computer Interface based on Forehead Multichannel
Biosignals to Control a Virtual Wheelchair. Paper presented at the ICBME08.
Goldberg, S. L., & Knerr, B. W. (1997). Collective Training in Virtual Environments: Exploring Performance
Requirements for Dismounted Soldier Simulation. In R. J. Seidel & P. R. Chantelier (Eds.), Virtual Reality,
Training s Future? (pp. 41-51). New York: Plenum.
Healey, J., & Picard, R. W. (2000). SmartCar: Detecting Driver Stress. Paper presented at the ICPR.00, Barcelona,
Spain.
Kim, K., Yoo, J., Kim, H., Son, W., & Lee, S. (2006). A Practical Biosignal-Based Human Interface Applicable to the
Assistive Systems for People with Motor Impairment. IEICE Trans. Inf. & Syst., E89-D(10), 2644-2652.
Lintern, G., Roscoe, S. N., Koonce, J. M., & Segal, L. D. (1990). Transfer of Landing Skills in Beginning Flight
Simulation. Human Factors, 32, 319-327.
Loftin, R. B., Savely, R. T., Benedetti, R., Culbert, C., Pusch, L., Jones, R., et al. (1997). Virtual Environment
Technology in Training: Results from the Hubble Space Telescope Mission. In R. J. Seidel & P. R. Chantelier
(Eds.), Virtual Reality, Training s Future? (pp. 93-103). New York: Plenum.
Magee, L. E. (1997). Virtual Reality Simulator (VRS) for Training Ship Handling Skills. In R. J. Seidel & P. R.
Chantelier (Eds.), Virtual Reality, Training s Future? (pp. 19-29). New York: Plenum.
Mahoney, D. P. (1997). Defensive Driving. Computer Graphics World, 20, 71-71.
Mahlke S. and Minge M. (2006). Emotions and EMG Measures of Facial Muscles in Interactive Contexts. ( not
published ).
114
9
th
Mastaglio, T. W., & Callahan, R. (1995). A Large-Scale Complex Virtual Environment for Team Training. Computer,
28(7), 49-56.
Moertini V.(2002) ,Introduction to Five Data Clustering Algorithm, Integral, Vol. 7, No. 2, October
Nasoz, F., Lisetti, C. L., Alvarez, K., & Finkelstein, N. (2003). Emotion recognition from Physiological Signals for
User Modelling of Affect. Paper presented at the the 3rd Workshop on Affective and Attitude user Modeling,
Pittsburgh, PA.
Neimenlehto P., Juhola M. and Surakka (2006), Detection of Electromyographic Signal from Facial Muscles with
Neural Networks, ACM Trans. Applied Perception, Vol. 3, No.1, January 2006, 48-61
Oskoei M. A. and Hu H. (2007). Myoelectric control systems- A survey. J. Biomedical Signal Processing and Control,
vol. 2, issue 4, 275-294.
Picard, R. W. (1997). Affective Computing. Cambridge, MA: MIT Press.
Psotka, J. (1995). Immersive Training Systems: Virtual Reality and Education and Training. Instructional Science, 23,
405-431.
Priyona A., Ridwan M. , Alias A. , Atiq R., Rahmat O.K., Hassan A., and Ali M. (2003). Generation of Fuzzy Rules
with Subtractive Clustering, Universiti Teknologi Malaysia, Jurnal Teknologi, 43(D) Dis. 143153
Regian, J. W., Shebilske, W. L., & Monk, J. M. (1992). Virtual Reality: an Instructional Medium for Visual-Spatial
Tasks. Journal of Communication, 42, 136-149.
Rezazadeh, I. M., Firoozabadi, S. M. P., & Hu, H. (2009). Facial Gesture Classification Using Multi-Channel Forehead
Bioelectric Signals for Human-Machine Interaction Applications. International Journal of Human Computer
Studies, Elsevier.
Schroeder, R. (1995). Learning From Virtual Reality Applications in Education. Virtual Reality, 1, 33-40.
Seidel, R. J., & Chatelier, P. R. (1997). Virtual Reality, Trainings future?: perspectives on virtual reality and related
emerging technologies. New York: Plenum Press
Simlog (Producer). (2004), Podcast retrieved from http://www.simlog.com/index.html.
Wakefield, R. R., OBrien, J. B., & Perng, J. (1996, December 16-19). Development and Application of Real-Time 3D
Simulators in Construction Training. Paper presented at the the International Conference on Construction
Training, Hong Kong.
Wilson, B. H., Mourant, R. R., Li, M., & Xu, W. (1998). A Virtual Environment for Training Overhead Crane
Operators: Real-time Implementation. IIE Transactions, 30(7), 589-595.

115
9
th

CONSTRUCTION DASHBOARD: AN EXPLORATORY
INFORMATION VISUALIZATION TOOL FOR MULTI-SYSTEM
CONSTRUCTION
Cheng-Han Kuo, Master Student,
Computer-Aided Engineering Group, Civil Engineering, National Taiwan University, Taipei, Taiwan
r96521601@ntu.edu.tw
Meng-Han Tsai, PhD Candidate,
d9521002@ntu.edu.tw
Shih-Chung Kang, Assistant Professor,
sckang@ntu.edu.tw; http://www.caece.net/sckang
Shang-Hsien Hsieh, Professor,
shhsieh@ntu.edu.tw; http://myweb.caece.net/shhsieh/index
ABSTRACT: This research examines an interactive construction information presentation tool called
Construction Dashboard, designed to explore inter-dependent information visualization. Construction
Dashboard allows personnel to comprehensively compare the situations of operational conflicts of services to
ease the decision-making process. This research developed a virtual multi-system building project to work with,
and conducted a user test with 30 participants by offering 9 practical problem related questionnaires (3 level
tasks) to see participants' problem-solving performances in both presentation modes. The results of the user test
indicate that Construction Dashboard can effectively save time and increase the accuracy of work. This
interactive approach of information display offers a means to explore, compare, and recognize the underlying
constraints of construction issues in a more profession manner.
KEYWORDS: Information Presentation Tool, Inter-Dependent Information Visualization, Multi-System, MEP,
Construction Dashboard
1. RESEARCH BACKGROUND
Inefficiencies in the visualization method used to display interrelated project information and the prevalence of
traditional paper-based methods have made the precise demonstration and individualized information of project
activities a difficult task (Liston et al. 2000; Fischer et al. 2002). This research aims to solve two types of
problems specifically in multi-system construction of a building; the first is to promptly solve conflicts among
service systems to sequence the work programs, and the second is to easily comprehend complex information in
a multi-system construction project (Fard et al. 2006).
To achieve the goal, this research proposes an interactive presentation framework tool for multi-system
constructions. Interactivity relies on visual data representations in a framework with construction scenarios,
which can in turn trigger a possible solution to avoid operational conflicts for execution in the practical field.
This is achieved through interactive data visualization, supported by a graphical interactive framework
mechanism that enables a complete, direct and meaningful presentation.
The proposed framework includes two presentation approaches (Flexible Information Layout and Information
Display Matrix), and four interactive methods (Visual Data Exploration, Cross-Highlighting, Time Controller
and Information Extraction. Based on these approaches, we have implemented an information display tool for
construction of multiple service network systems in building, which is named Construction Dashboard.
Construction engineers can examine the construction information through direct manipulation and get
immediate feedback from Construction Dashboard, due to the dynamic interrelation of data. Therefore,
engineers can understand the interdependencies of construction information of different service systems, and
deal with the potential problems in advance. In this work, we address the problem of direct data interaction,
focusing particularly on the case where interactive presentation is most important.
117
9
th

2. INFORMATION IN MULTI-SYSTEM CONSTRUCTION
This research aims to focus different modes of information presentation in the construction process for
coordination of multi-system construction and understanding of massive interdependencies of construction
information. We propose four kinds of critical information in this respect:
Temporal Information: The temporal information is an essential factor in construction ground that
is widely used in project planning and management. A project manager has to depend on temporal
information to control and monitor progress of a project with a view to completion by the scheduled
time. To represent temporal information, this research used the Gantt chart, as generally used in
construction management.
Spatial Information: The spatial information includes geometric information of the assembly, route
and installation positions of the construction elements. To achieve high building performance in a
multi-system construction project, construction engineers usually use spatial information for
interference checking of distribution systems, paths, circuits, ducts, fittings, shapes and
configurations, feeders, panels, and so forth of MEP works to solve collision problems. The layouts
and positions of construction elements of a multi-service system are of a composite character as they
pass through, sustained or are placed closely to each other, or cross, overlap or clash in multiple
places. Consequently, conflict problems occur frequently in multi-system constructions, making
spatial information one of the most important forms of information for multi-system construction.
Presently, 3D models and rendering are widely used to represent spatial information. This research
followed these conventions.
Hierarchical Information: The hierarchical information includes grouping manners, and the
hierarchical relationship of work items. It can help a construction manager understand the scope,
responsibility and organization of work items. In the multi-system constructions, a project manager
needs to clearly understand the classification and organization of each system in order to efficiently
control the project and thereby achieve high building performance. This research uses tree diagrams
because they effectively represent the hierarchical relationship and organization of information, and
are easily readable for less-experienced users.
Relational Information: The relational information includes a number of relationships established
from the trial run of work items and system elements. In the trial run phase, systems usually need to
cooperate and mutually support each other. For example, a water supply system requires a power
system to provide power to run its motor. Similarly, AC systems are in principle dependent on a
power and water supply system. Hence, if the project manager cannot understand the relationships
between trial run work items and system elements, it will lead to project delays. We used a network
chart because its nodes and edges can represent the relationship of work items in an easily
comprehensible way.
3. PROPOSED PRESENTATION METHODS
Appropriate information presentation methods are the perfect cognitive process that can help users to browse,
explore, and compare information at pre-construction, construction and post-construction phases of a project.
Hence, this research proposed two presentation methods of multi-system construction:
Flexible Information Layout: Flexible information layout can let users modify view-size and the
position of information according to its importance and weight. Therefore, users can easily discover,
explore, and compare particular information to concentrate on one or more specific piece of
information that they need. For these reasons, we adopted flexible information layout technology in
developing the presentation skill of Construction Dashboard.
Information Display Matrix: Because of the limitations of monitor size and computer resolution, we
usually combined several monitors in a matrix form to display multi-information by keeping the
quantity and visibility of information the same. The Information Display Matrix can ensure that all
information has at least the same display area and links with each other; users can then freely move
information objects in any direction, such as up to down, left to right, or vice versa.

118
9
th

4. PROPOSED INTERACTION METHODS
Information interaction approaches can increase the visibility and readability of information so users can browse,
and diagnose information effectively. Hence, this research proposed four interactive approaches of multi-system
construction:
Visual Information Exploration: In order to users can easily explore information, this research
proposes three kinds of visual information exploration: zooming, panning, and rotation from visual
information seeking mantra (Shneiderman, 1996).
Cross-Highlighting: Cross-highlighting can represent the relationships between different information.
Therefore, it can help users in describing, explaining, and comparing information about a construction
project, leading to improvement in the decision making process (Liston and Fischer, 2000). For this
reason, this research proposed a cross-highlighting function to represent the relationships between
different information.
Use of Time Controller: This research proposes a time controller, which integrates other
construction information. Users can use this time controller to manipulate temporal information to
understand the relationship among different kinds information. This research also introduces the use
of different colors that takes place with the operation of the time controller to represent differences in
construction status.
Information Extraction: Information extraction approaches include information filtering,
information emphasizing, and information searching. Information filtering can let users hide
irrelevant information to help show only significant items. Information emphasizing can let users pick
up and display significant information, and at the same time hide unnecessary information.
Information searching can let users view relevant items that comply with keyword commands given
by users.
5. IMPLEMENTATION OF A CONSTRUCTION DASHBOARD
5.1 System Architecture
The overall composition of Construction Dashboard is illustrated in FIG. 1. It has a three-layer structure
comprising a user-interface layer, a data-process layer, and a data-storage layer. Each layer is composed of
major components or functions, which are represented by blocks. The arrows between each layer represent the
direction of communication or flow of data. In this design format, the user-interface layer is responsible for
providing functions so that users can manipulate and interact with elements on each view directly to present the
results in a visual way. The data process layer is responsible for extracting data from the database and producing
the view table needed to generate elements on each view based on users requirements. The data storage layer is
responsible for storing various construction data.
During software design, users can manipulate views and elements on each display directly, and get visual
feedback in the manipulating process simultaneously. The following paragraphs describe each feature of the
Construction Dashboard separately.

FIG. 1: Three Layer Structure of Construction Dashboard
119
9
th

Views: The views are responsible for presenting construction information in a cognitive way. Each view
extracted required data from the view table to produce elements according to the information required to
display, and then presents and arranges the elements based on the layout algorithm of view.
Manipulator: Users can manipulate the views and elements with two major functions of Construction
Dashboard. One is visual interactive functions and the other is cross-view functions. When a user uses
visual interactive functions, such as zooming and panning of view, the manipulator will maneuver the view
directly. When a user uses the cross-view functions, such as cross-highlighting, information extraction, and
information searching, the manipulator will search other views elements, which are using the same row of
the view table, and start the corresponding functions.
View Table: The view table is responsible for providing data to each view. Each column of the view table
represents item-wise construction data, and each row represents property-wise construction information.
Data Processor: The Data Processor is responsible for extracting data from the database to update the view
table to produce elements on each view, as per users requirements, and activate cross-view functions.
5.2 Information Presentation in Construction Dashboard
Construction Dashboard is designed with two information presentation skills, the flexible information layout,
and information display matrix. The following paragraphs explain each presentation function of the
Construction Dashboard separately.
Flexible Information Layout: As shown in Fig. 2, all information layout views of Construction
Dashboard are floating windows. Hence, users can adjust the size and position of each view according to
the requirements of viewers.
Information Display Matrix: As shown in Fig. 3, this research used four 24 LCD monitors and arranges
them in a matrix style, so that users can easily regulate information pages up to down, left to right and vice
versa.

FIG. 2: Flexible Information Layout

FIG. 3: Information Display Matrix
120
9
th

5.3 Information Interaction in Construction Dashboard
Visual Information Exploration: As shown in FIG. 4, it can perform Zooming, Panning, and Rotation of
displays (Information views). As Construction Dashboard emphasizes vector graphics, the user can
minimize or enlarge elements of any view with a clear and smooth look, keeping the screens resolution or
DPI setting unchanged.
Cross-Highlighting Technique: As shown in FIG. 5, when a user selects elements from any view of
Construction Dashboard, the elements will change its color to green, and other relevant elements of that
selected element will change their color to dark-blue. For example, when a bar of the temporal
information view is selected, the related 3D object of spatial view, the nodes of hierarchical view, and the
items of relational information view, will change their colors. Thus users can observe and understand the
interdependencies between construction elements easily.
Time Controlling Maneuver: As shown in FIG. 6, after opening the time controller window, the element
of each view will change color to represent the construction status according to its construction start time,
finish time, and the current time of the time controller. Construction Dashboard uses three different colors
to represent the construction status of work items: gray represents the element that has not been started yet,
orange represents the element under construction, and magenta represents the element already completed.
Information Extraction Role of Construction Dashboard: Construction Dashboard can perform three
functions i.e., information filtering, information emphasizing, and information searching, to exploit its
information extraction performance. Information filtering, as shown in FIG. 7, in the hierarchical
information view, is where the construction director can use a filtering function to hide irrelevant nodes
for the purpose of showing the significant node(s) as per users demand. Similarly, Construction
Dashboard, in another view of the display matrix, can also conceal or reduce particular element(s) through
a cross-highlighting function to show the importance of a particular element. Information emphasizing, as
shown in FIG. 8 is where users like to observe the relationship among nodes in the relational information
view. The users only need to put the mouse over the desired node, and then Construction Dashboard will
only show the relevant nodes. The brightness and colors of other irrelevant nodes will diminish. Thus
irrelevant nodes disappear from the screen. Information searching, as shown in FIG. 9, allows users to
select any list of items to search in the searching window, and then Construction Dashboard will highlight
the objects and relevant information in each view by using the same data.
FIG. 4: Visual Information Exploration

121
9
th
FIG. 5: Cross-Highlighting
FIG. 6: Time-Controlling Maneuver
FIG.7: Information Filtering

122
9
th
FIG. 8: Information Emphasizing
FIG. 9: Information Searching

6. EXAMPLE CASE OF A MULTIPLE SYSTEM CONSTRUCTION
We created an example case to validate the practicality of Construction Dashboard. We built a virtual building
project as an example case in a multisystem computer room, which has 154 elements in the 3D model and a
project duration of approximately 100 days. Here we intended to verify how effectively Construction Dashboard
can help users explore and understand relevant construction information. We also investigated whether users can
effectively identify the problematic work areas in a user test by exploring information from Construction
Dashboard. The details are described as follows.
In this example case, we refer to a project to build a 3D model with the help of a multi-system computer room in
the Faculty of Civil Engineering building at National Taiwan University. The example case involves a water
system, power system, air-conditioning system and computer system (FIG. 10).

FIG. 10: Example Case
123
9
th

7. USER TEST
7.1 Test Plan
As shown in FIG. 11, the test plan followed the 233 arrangement (2 is for exhibit manners, 3
denotes the tasks to perform and 3 is the three levels of questionnaires). All users needed to perform
two exhibit manners; i.e. presentation through Construction Dashboard and presentation through
paper-based media. In the test, users were required to perform three tasks, each of which included
three levels of questions to answer. The following sections explain the test plan in more detail.

FIG. 11: User Test Procedure
7.2 Test Participants
There were 30 participants in the user test, comprised of 21 males and 9 females. Their ages ranged from
twenty-three to thirty-four years. The participants included 8 civil engineers and 22 graduate students from a
civil engineering background. All participants had studied construction management related courses.
7.3 Test Results
An level of 0.05 was used for all statistical tests and analysis. The test results assessed how quickly and
accurately participants performed the task when using Construction Dashboard. Thus we measured the
efficiency and success rates of Construction Dashboard. They are summarized as follows:
Efficiency: The statistical results, including mean value and standard deviation for five dimensions are
shown in FIG. 12. All of the tasks were considered significant (p<0.05) in t test. Users of Construction
Dashboard spent less time than required when using paper-based media.
Success Rates: The success rates are shown in FIG. 13. From the figure, we can find that Construction
Dashboard results in higher averaging successful rate in eight out of nine testing tasks. We also used
statistical methods, t test and chi-square test, to see the difference between groups. We found that three of
the tasks, including Equipment-Coordination, Equipment-Projectwise and Function-Projectwise, reach
statistical significance.

124
9
th

FIG. 12: Test Results of Efficiency

FIG. 13: Test Results of Success Rates
8. FINDINGS FROM THE USER TEST
Overall, the test results were positive regarding the use of Construction Dashboard as a presentation tool for
interactive information presentation. From the user test we found conclusive evidence in four aspects:
We used a matrix style for presentation of Construction Dashboard. Most users deemed it helpful when
comparing different information at a time, thus reducing the difficulty of memorizing more data. A minority
of users opined that they were tired of raising their heads to look over information.
We used a flexible information layout for Construction Dashboard. All users agreed that this made their
125
9
th

work more convenient. The flexible information layout function is accorded with good presentation
phenomenon, and is compatible and favorable for presentation purposes that users might feel it is easy to
operate.
All users agreed that the interaction mechanism to present interdependent information was better than
before. They felt that Construction Dashboard could improve the cognitive power required to catch
construction related information more precisely.
In the test, we found that when users performed the paper-based presentation questioners first, most users
used the same method to answer the questions for the test of Construction Dashboards presentation. Most
forgot the function of Construction Dashboard in the test. Although we taught them and let them practice
before the test, the time of tutoring and practice was too short to remember some important functions of
Construction Dashboard.
9. CONCLUSIONS
We developed an interactive construction information display tool called Construction Dashboard on top of the
traditional paper-based mode of presentation. Based on this tool, users can explore, manipulate and compare
construction information through interactive visualization, and then make decisions in a more pragmatic way to
avoid collisions of construction systems. This research explored realistic information presentation and
interactive approaches to improve users' cognition of the relationship among critical construction information,
which enhances the communication of design intent to clients and helps avoid costly redesign onsite. We used a
multi-system computer room for our experiment, and designed a series of questions for our user test to compare
Construction Dashboard with the traditional presentation mode. In the test result, we discovered Construction
Dashboard can help users to explore and understand the inter-dependencies and underlying meaning of
information, thereby making it possible to solve problems more efficiently. Thus users are able to predict, solve
and answer questions more efficiently, after minimizing design coordination errors.
10. REFERENCE
Fard, M. G., Staub-French, S., Po, B., and Tory, M. (2006). Requirements for a Mobile Interactive Workspace
to Support Design Development and Coordination, Joint International Conference on Computing and
Decision Making in Civil and Building Engineering, Montral, Canada, 14.-16. June 2006
Fischer, M., Stone M., Liston, K., Kunz, J., and Singhal, V. (2002). "Multi-stakeholder collaboration: The CIFE
iRoom," Proceedings CIB W78 Conference 2002: Distributing Knowledge in Building, Aarhus School of
Architecture and Centre for Integrated Design, Aarhus, Denmark, 12.-14. June 2002
Liston, K., Fischer, M. and Kunz, J. (2000). Designing and Evaluating Visualization Techniques for
Construction Planning, The 8th International Conference on Computing in Civil and Building Engineering,
Stanford, CA, USA.
Liston, K., Kunz, J., and Fischer, M. (2000). Requirements and Benefits of Interactive Information Workspaces
in Construction, The 8th International Conference on Computing in Civil and Building Engineering,
Stanford, CA, USA.
Shneiderman, B. (1996). The Eyes Have It: A Task by Data Type Taxonomy for Information Visualization,
Proc. Visual Languages, pp. 336-343.
126
9
th

COMPUTER GAMING TECHNOLOGY AND POROSITY
Russell Lowe, Senior Lecture in Architecture,
University of New South Wales, Faculty of the Built Environment;
russell.lowe@unsw.edu.au , www.russelllowe.com
Richard Goodwin, Professor of Fine Art,
University of New South Wales, College of Fine Art;
richard@richard-goodwin.com , www.richard-goodwin.com
ABSTRACT: In 1996 artist-architect Richard Goodwin coined the term "Porosity". Porosity describes the publicly
accessible spaces within privately owned parts of the city. Any mixed use building is necessarily Porous; for
example, clients must be able to visit their dentist's surgery on the 14th floor, their lawyer on the 5th floor, or a
restaurant on the roof. A buildings Porosity is a measure of the quantity and quality of pathways to a given
destination (Goodwin 2006).
More recently, the growing list of urban mapping projects suggests that there is an urgent need for a deeper
understanding of the dynamic relationship between public access and the occupiable spaces of the city (see Reades
et al 2007, for a representative range of these, C. Nold's work is worth a special mention). The Porosity of a
building is an excellent example of the dynamic relationship between people and the built fabric of the city. Due to
the manual data gathering techniques employed, the first incarnation of the Porosity maps were only able to create
a 'snapshot' of the buildings selected. To understand how the Porosity of a specific building might change over time
the mapping process would need to be automated.
The questions that initiated this research were, "could Porosity be represented in real time? What should that
representation look like? And can the combination of computer gaming technology and environmental sensors
automate the representation of Porosity?"
In response to these questions the authors have developed a prototype that translates the movement of a person in
the real world into the virtual environment of a computer game; note the pedestrians' participation is entirely
passive (i.e. they are not knowingly playing a computer game, they are simply going about their business). The
movements of a Non-Player Avatar, standing in for the pedestrian, are then represented with a range of textures,
geometries and behaviors. (The external sensor that is being used to demonstrate proof of concept is the Nintendo
Wii Balance Board, employing a custom script to interface with the PC). The authors call these representations of
movement and time 'Porosity Lenses'. Their development draws from Goodwin's Porosity Index but, significantly,
construct it in real time. In one lens the movement of the avatar constructs a facsimile of a space as sensors
passively capture a person's movement through the real one.
Finally the paper compares the lenses developed with recent representations of movement over time to highlight
strengths and weaknesses of the approach.
KEYWORDS: Porosity, Computer Games, Sensors, Representation, Mapping.

1. INTRODUCTION
A growing list of urban mapping projects suggests there is an urgent need for a deeper understanding of the dynamic
relationship between public access and the occupiable spaces of the city (see Reades et al 2007, for a representative
range of these, C. Nold's work is worth a special mention). In many cases these projects represent dramatically
changing patterns of use, mobility, and security. The term "Porosity", coined by Richard Goodwin, describes the
publicly accessible spaces within privately owned parts of the city. With the support of an Australian Research
Council Discovery Grant from 2003-2005 Goodwin and his research team mapped these "Porous" spaces within the
Sydney CBD. The results suggest new opportunities for pedestrian movement through the city. In contrast to many
127
9
th

of the urban mapping projects cited above the Porosity maps are fully three dimensional. By recording the member
of the public's duration of stay they also capture the dimension of time. However, due to the manual data gathering
techniques employed, the first incarnation of the Porosity maps were only able to create a 'snapshot' of the buildings
selected. To understand how the Porosity of a specific building might change over time the mapping process would
need to be automated. This prompted questions such as can Porosity be represented over time, and ideally in real
time? What should that representation look like? Can the combination of computer gaming technology and
environmental sensors automate the representation of Porosity?'
In this paper the authors describe a new way to map the Porosity of a building by modifying an off-the-shelf
computer game, Unreal Tournament 3 (UT3) by Epic games, and using sensor controlled "Non-Player Avatars". In a
typical single player computer game the player knowingly controls an "Avatar" (which is the players embodiment
within the virtual world) and may compete against or be assisted by Non-Player Characters (NPC's) which are
controlled by the computer game's artificial intelligence. In a multiplayer game the player is usually competing
against or being assisted by Avatars that are knowingly controlled by other real people. In contrast to these typical
situations the authors have created a prototype where a real pedestrian's presence in the virtual environment is
entirely passive i.e. they are not using the computer or knowingly playing a computer game, they are simply going
about their business. This also contrasts Gemeinboeck et al's (2005) approach where the "spatial evolution
unfolds in the mutual interplay between the participant and the virtual opposite." To clarify this distinction the
authors have coined the hybrid term "Non-Player Avatar" or NPA. The movements of an NPA are driven by sensors
recording a pedestrian's movement within a real environment and simultaneously traced in virtual space and time
with a range of textures, geometries and behaviors. The authors call these representations of movement and time
'Porosity Lenses'.
The Porosity Lenses are designed to facilitate analysis, by Avatars within the virtual environment, of NPA
movement from many different points of view thereby taking advantage of Sun et al's (2007) research that notes the
importance of three dimensional space and point-of-view in shaping human behavior in urban spaces. In an
extension to Sun el al's work the authors propose that if three dimensional space and the first person point-of-view is
important in shaping human behavior in urban spaces then they may be equally important in understanding and
analyzing human behavior in these situations.
This paper will describe the theoretical context, design and development of the Porosity Lenses, while comparing
and contrasting them to recent representations of movement over time. In one example the movement of an avatar
generates three dimensional building blocks that construct a facsimile of the pedestrian's environment in real time as
sensors record their movement through it. It will also describe the development and preliminary testing of a
prototype sensor solution that uses off the shelf computer gaming hardware.
As noted above there are many examples of projects that map the movements of people in cities and the developers
of Halo 3 (Bungie Studios in collaboration with Microsoft) have mapped and analyzed over 3,000 hours of game
play (Thompson 2007). The research presented here uses off the shelf computer gaming software and hardware with
a view to combining and extending the two approaches.
2. THE POROSITY STUDIO.
In 1996 artist-architect Richard Goodwin established the Porosity studio within the College of Fine Arts (COFA) at
the University of New South Wales. Goodwin notes that the public space of the city doesn't end at the building
envelope; that any mixed use building requires access by the public and is necessarily porous.
For example, clients must be able to visit their dentist's surgery on the 14th floor of a building, their lawyer on the
5th floor, or a restaurant on the roof. A buildings Porosity is a measure of the quantity (and quality) of pathways to a
given destination (Goodwin 2006). The primary concern of Goodwin's research was the amount of time that the
Porosity researcher could spend within a privately owned building without detection. After comprehensive
fieldwork detailed three dimensional maps of three major zones within the Sydney Central Business District (CBD)
were produced. Architectural data on the buildings within those three zones, combined with the results of the field
work, gave each building a qualitative Porosity Index. In the original Porosity Index Goodwin cited orientation,
duration of stay, adjacency to lifts, stairs and other distinctive architectural qualities as factors that contribute to a
building's Porosity Index.
128
9
th

Goodwin proposed that reading these Porosity Indexes could give planners and architects direction as to the way in
which new linkages may be made which enhance the public space in the city. But from a slightly more sinister point
of view the Porosity Index's can also measure and qualify the dilemmas of security versus access in relation to
public and private space. In other words the Porousness of a building relates to the ease by which a building might
be accessed and evacuated; the irony here is that high levels of Porousness would seem to facilitate both.
3. THE APPROPRIATENESS OF USING COMPUTER GAMING TECHNOLOGY TO
REPRESENT POROSITY IN REAL TIME.
Rather than being fully spatial many urban mapping projects still represent the city in two dimensions. A few
projects supplement those dimensions with other, non-spatial, dimensions such as information about the users
itinerary (E. Polak's "Amsterdam RealTime: Diary in Traces") and physiological responses (C. Nold's Biomapping
project, 2004 - ongoing). In "The Language of New Media" (2000) Lev Manovich says that "along with providing a
key foundation for new media aesthetics, navigable space has also become the new tool of labor." He goes on to say
that "the 3-D virtual space combined with a camera model is the accepted way to visualize all information".
Demonstrating the pragmatic advantage of visualizing information in this way Sun et al (2007) recognized the
importance of point-of-view in shaping human behavior in urban spaces. By using a head cave and three
dimensional virtual environments they found that some assumptions about human behavior in urban spaces could be
challenged. It's interesting to note here that their research utilized an environment that was "designed to be
something like a first person shooting game, such as DOOM." In contrast the authors did not use something like a
first person shooting game, they actually used an off the shelf computer game. So why take Sun et al so literally and
use an off the shelf technology designed in the first place for entertainment? The answer lies in the underlying
sophistication of computer games and, more recently, their versatility regarding modification; not to mention the
widespread encouragement by game developers and large "modding" communities that users create custom game
dynamics and content.
Microsoft Research wrote in 2005 that computer game technology "pushes the technology envelope". Also
recognising this, the game developer "Virtual Hero's" (the developer of the groundbreaking simulation/marketing
tool "Americas Army") has recently licensed the UT3 game engine to develop an urban training simulation called
Zero Hour: Americas Medic. Illustrating the growing institutional acceptance of repurposing entertainment
technology Virtual Hero's are also working in collaboration with the American Department of Homeland Security.
In 2006 Price provided a useful summary of the recent "deployment of game engine technology" for education and
training applications, noting that their use in such ways has occurred only in the "last few years". In 2000
Bouchlaghem et al noted that "the benefits and applications of virtual reality (VR) in the construction industry have
been investigated for almost a decade" but that "the practical implementation of VR in the construction industry has
yet to reach maturity owing to technical constraints". Ten years on from Bouchlaghem et al's study we find that
industry heavyweight Autodesk has recently become a member of the "Integrated Partners Program" with EPIC
games and, as Author 1 has mentioned in a previous paper, the use of computer gaming technology in education
reflects moves in practice by large firms such as Texas based HKS and small by Lara Calder Architects, Sydney. In
a move that represents an alternative way forward to their main Building Information Modeling (BIM) competitor
(Autodesk's Revit) Graphisoft's Archicad has incorporated a real time interactive engine in its latest release. While
the authors applaud the initiative they remain skeptical that Graphisofts resources and culture will facilitate their
engines comprehensive development (or importantly, many alternative developments). The use of VR technologies
in the construction industry is still not mature, but many of their technical constraints have been overcome
somewhat ironically, by the entertainment industry.
In the discussion relating to human behavior in public spaces Sun et al state their assumption that "from the set of
architectural clues in sight, the human selects the one with the highest priority and performs a related strategy"
(2001). To reiterate, in the context of the authors project as humans perform navigational strategies within a real
environment NPA's mirror them within a virtual environment. The Porosity Lenses described below are mechanisms
that add persistent traces of movement to the "set of architectural clues in sight" so that Avatar analysts can develop
an understanding of the movement of NPA's; and by extension the pedestrians that are driving them. By utilizing
computer games the Avatar analysts can take advantage of the "positive benefits of video game play" that include
"spatial visualization and mental rotation" (Rosser et al, 2007).
129
9
th

4. DESIGNING 'POROSITY LENSES' USING UNREAL TOURNAMENT 3.
Representing one of the latest generation of computer gaming technologies, Unreal Tournament 3 (UT3)
incorporates an incredibly comprehensive tool set and has the support of a large game moding community (see the
forums at www.3dBuzz.com and http://forums.epicgames.com/forumdisplay.php?f=335 for example). For these
reasons UT3 was chosen as the computer game medium within which to develop the initial functional prototypes of
the Porosity Lenses. The UT3 world editor, UnrealEd, "is a suite of tools for working with content in the Unreal
Engine. At the core, it is used for level [virtual environment] design; but contained within are editors and browsers
for importing and manipulating content for your game project (EPIC Games)." The toolsets used to design and
develop the Porosity Lenses include UnrealKismet, Matinee, Cascade, the Material and Static Mesh Editors. Many
of the toolsets require information generated in third party software (textures or geometry for example) and from
other toolsets within the editor itself. While this interdependence adds to the complexity of modifying the game it
does provide many opportunities to link different types of parameters and contributes to the sophistication of the
interactivity.
For the first Porosity Lens the UnrealKismet toolset was used in conjunction with the Cascade toolset to attach,
detach and control the emission of a sprite particle emitter that was attached to a NPA (a sprite is a 2d surface that
always faces the player). UnrealKismet "allows non-programmers to script complex gameplay flow in level. It
works by allowing you to connect simple functional Sequence Objects to form complex sequences (EPIC Games)."
In this and the following examples visual scripting was used to create a mechanism for interactivity that didn't exist
previously within the UT3 game. The result is that as the NPA moves around the environment it leaves a trail of
translucent squares that traces its movement through space and time; much like the breadcrumbs left by Hansel and
Gretel in the well known fable, figure 1. The custom material applied to the sprite contains a variable opacity-
parameter so that the translucency can be adjusted to balance between the clarity of the avatars path and the density
of its representation. The density of the path at any one point represents the overlaying of multiple translucent sprites
that build opacity and represent the duration spent at that point; a key factor to understanding Porosity.
Figure 1. Shows the results of the Kismet sequence created by the authors that controls the relationship between a
particle emitter and an NPA. The sprite is a translucent red square which is emitted at a rate of 10 instances per
second. The effect is much like the breadcrumbs left by Hansel and Gretel in the well known fable but in this case
records both the path taken and the duration spent at any point along it. Avatar analysts are also able to
dynamically control the translucency of existing structures within the environment.
Extending from the notion of parametric interdependence mentioned above, every element within the game
environment contains parameters that can be customized. Identifying parameters and referencing them within
Kismet enables many to be changed over time. These changes can be pre-scripted (much like conventional
animation) or can occur in real-time. For the second Porosity Lens a Kismet sequence was designed that modified a
130
9
th

scalar parameter which controls the opacity channel of a material applied to a rectilinear block within the
environment. The collision properties of the block are set to register "touch events" with NPA's but will not collide
with them physically, so that they do not impede their progress. Every time a block is touched by an NPA its opacity
drops by 0.2 (where 1.0 is opaque and 0 is transparent). The sequence continues reducing the resulting opacity, upon
subsequent touch events, until it reaches zero; i.e. until the material is completely transparent. Figure 2. shows a
simple arrangement of corridors filled with the blocks.
Figure 2. Shows the blocks in a corridor. Their opacity is controlled by a Kismet sequence linked to one of their
materials properties, each time a "touch event" occurs the opacity of the block is reduced by 0.2, until the block is
completely transparent. This gives the impression of the NPA or NPA's slowly carving out the space that they
occupy.
Multiple touches by one or more NPA's give the impression of their movement slowly "carving out" the space of the
corridors. One can see that as long as walls, floors, ceilings and doorways limit the pedestrian's movement in their
real environment there would be no need to represent them within the virtual environment; the presence or absence
of rectilinear blocks can perform this role. The authors imagine an environment totally filled with these blocks in a
complete 3d matrix. The virtual representation of space that is occupied by pedestrians in a real environment would
become clear as the NPA paralleling their movements traveled through it.
The third Porosity Lens adopts the additive approach of the Hansel and Gretel Lens with the 3d geometry of lens
two. In contrast to the Hansel and Gretel Lens however the mesh is only emitted when the avatar "carrier" is
moving; i.e. it records position rather than position and duration. The emitter could be set to emit mesh particles
constantly, but the additional processing required by 3d geometry over a simple sprite slows the computer
significantly when count rises into the thousands. At the current stage of its development the mesh elements do not
collide with the NPA. Ultimately the authors intend that the mesh will collide with the footsteps of the NPA so as
the pedestrians negotiate a real environment they passively construct a version of it beneath their feet in a virtual
environment. See figure 3.
In the three lenses described above two representational strategies are employed; the sprites and tiles are additive,
the rectilinear blocks are subtractive. Further, each of the prototypes utilizes arbitrary rectilinear shapes or geometry
and the textures used are homogonous. As working prototypes they demonstrate that the answer to the question "can
Porosity be represented in real-time?" is yes, but there is clearly room for improvement. The following section looks
at three examples of urban mapping projects that highlight strengths, weaknesses and suggest possible directions
regarding future development for the Porosity Lenses.
131
9
th

Figure 3. Real-Time construction of an environment. Blocks emitted by the NPA as they move around an empty
virtual environment would construct a facsimile of the pedestrians real environment. Note the image above shows
blocks leaping over ones previous laid avoiding an intersection; in those cases the NPA literally leaped over the
earlier path. This demonstrate that environments that change in the vertical dimension (stairs, ramps, etc) are able
to be replicated .
5. WHAT SHOULD THE POROSITY LENSES LOOK LIKE?
In the following examples the authors critically examine recent efforts by various researchers to represent the
movement of people through space.
In C. Nold's work, seen at www.biomapping.net, "participants are wired up with an innovative device which records
the wearer's Galvanic Skin Response (GSR), which is a simple indicator of the emotional arousal in conjunction
with their geographical location. People re-explore their local area by walking the neighbourhood with the device
and on their return a map is created which visualises points of high and low arousal." The resulting maps of
Stockport, Greenwich and San Francisco are two dimensional. The map for San Francisco uses stacked red disks and
at first appearance bears some similarities with the sprite based Porosity Lens. This similarity is short lived however
as the colour intensity isn't built up in layers but comprises arbitrary steps on a scale; each disk is opaque. A sample
video on www.biomapping.net shows a three dimensional structure built over Google Earth.
Figure 4. Screen capture from www.biomapping.net showing C. Nold's mapping of "emotional arousal in
conjunction with geographic data." The effect is similar to a line graph, folded over a surface in Google Earth. Note
the horizontal surface is completely flat and that any rise in elevation could confuse the reading of data in the z-axis.
132
9
th

The line in the xy plane traces the participant's movement through the environment while the z-axis is used to
represent emotional arousal. The effect is similar to a line graph, with solid fill beneath, which is folded so that it
might stand up unsupported. Figure 4. The height in the z-axis plays the same role as the density of the Hansel and
Gretel sprite trail, but this approach would quickly become confusing if the ground plane was not completely flat
(i.e. a point on the terrain, or within a building, with a higher elevation could give the impression that the experience
there is more intense). Expressing the data as a cross section area perpendicular to the direction of movement may
alleviate that confusion and suggests a future direction of research for the authors.
E. Polak et al's project Amsterdam RealTime tracks people equipped with GPS enabled devices in real time and
projects the resulting lines onto a black background. The map is 2 dimensional but expresses a direct precursor to a
key strategy of the Porosity Lenses (using people movement to build an environment); as explained by the Waag
Society "this map did not register streets or blocks of houses, but consisted of the sheer movements of real people."
The time lapse animation shown here: http://www.waag.org/project/realtime shows the map glowing and pulsating
at points where multiple pathways cross. The representation of intensity is unmistakable qualitatively but vague in a
quantitative sense. This may be a strength and weakness of all strategies that rely on multiple 2 dimensional layers.
Bungie Studios the developers of Halo 3 (in collaboration with Microsoft) have developed tools to extract gameplay
data so they could map and analyze over 3,000 hours of game play (Thompson 2007). In one example "player
deaths [are] represented in dark red on [a] 'heat map' of the level". Alison Mealey manipulates a similar approach to
create portraits by recording and representing the movements of NPC's in the game UT2004 (Petersen 2005).
Another example from Bungie shows "superimposed locations of about 30 testers after half an hour of gameplay";
players left different "coloured dots showing player location at five-second intervals (each colour is a new time
stamp)". When the dots were clustered by colour it demonstrates that "players were moving smoothly through the
map". Currently the Porosity Lens's don't implement a similar facility; not only would it show consistency of
movement through an environment it would also confirm the direction of that movement. Once again these
examples are 2 dimensional, but the final example from Bungie shows the importance of understanding actions from
a player's perspective in 3 dimensions. In this example Bungie's analysts noted a high rate of "suicides" in a
particular area of a map; the "heat map" would show them the location but the action in this case was not shaped
environmentally but locally. "The players were firing the tank's gun when its turret was pointed toward the ground,
attempting to wipe out nearby attackers. But the explosion ended up also killing (and frustrating) the player"
(Thompson 2007). In this example 2 dimensional orthographic views and real-time spatial experience combine to
give a more complete representation of the causes and effects of a user's navigation through an environment. See
Figure 5.
Figure 5. On the left is a 'Heat Map" showing the number of player deaths at particular points within the
environment. The map on the right shows specific colours marking player location at five-second intervals. This is
one strategy for confirming direction of movement; as well as clearly marking trends and those opposing them.
Within these examples three major issues arise; the qualitative vs quantitative advantages of utilising Colour
Intensity and Size as mechanisms to represent duration of stay in any one place; the obscuring of data by either
subsequent entries by the same person or by entries from another person; and the representation of the "activity
workspace" (Mallasi, 2004). The obscuring of data is seen in Nold's work (Figure 6) and in the authors (Figure 4)
and would be a consideration in the development of a cross sectional area representation mentioned above. Currently
the authors are able to adjust the translucency of elements within the UT3 environment in real time; extending this
functionality to the Porosity Lenses themselves may prove beneficial.
133
9
th

In a paper regarding construction activities' workspace conflicts Mallasi (2004) sees that "to specify the workspace
requirement in a dynamic way, while satisfying a set of spatial dynamics and change of workplace over time
intervals, is a difficult problem". To mitigate this problem in his research he uses a technique that represents the
activity workspace in a series of 3dimensional boxes. Each box defines a workspace (such as above, below, or
surrounding) that is a "generic capture of different workspace requirements." While the second Porosity Lens
similarly utilises rectilinear blocks it indexes them to the size of the construction worker him or herself. By doing
this a more fine grained understanding of the activity workspace would result (see Figure 2).
6. THE NINTENDO WII "BALANCE BOARD" AS AN EXTERNAL SENSOR
LINKING PEDESTRIAN MOVEMENT TO UT3 NON PLAYER AVATAR
MOVEMENT; AUTOMATING THE COLLECTION OF DATA.
In a typical computer game the avatar translates the actions of real people into the virtual environment.
Conventionally one controls their avatar directly and might use a computer keyboard and mouse or gamepad to do
so. In addition the computer controls various other characters within the game environment with artificial
intelligence. A third category, sensor controlled avatars, has recently emerged. These avatars are controlled by
sensors that pick up the movement of a person in a real environment and translate it to a virtual environment.
Groenda et al (2005) use tracking and Motion Compression to allow the exploration of "an arbitrarily large virtual
environment while the user is actually moving in an environment of limited size". While Motion Compression
would be useful for play in gaming halls or at home it takes this tracking technology in the opposite direction to the
authors' project that uses sensor controlled avatars for mapping real spaces. In other words, while Gronenda et al
saw that the virtual environment would be "limited to the size of the user's real environment" the authors see this as
an opportunity to map the limits of a real environment by tracing pedestrian movements through it.
Devices for tracking people through environments include GPS (Global Positioning Systems) INS (Inertial
Navigation Systems) and Radio Frequency (RF) based positioning systems. Due to the particular challenges of
urban environments none of the systems listed above offer a comprehensive solution. A hybrid system is required.
Allen et al note the "widespread availability" of low-cost computer game peripherals and see an opportunity to
"adapt technology designed for the entertainment industry and develop low-cost measurement tools for use in
clinical science and rehabilitation engineering (2007)." A major advantage for Allen et al is to break free from the
limits of the clinic; a key limit being expensive equipment. For the author's the major advantage is that with
"widespread availability" ultimately there may be a good chance that a pedestrian might already be carrying the
sensor we could use to track them.
With the notion of repurposing off the shelf computer game peripherals in mind the authors tested a Nintendo Wii
"Balance Board" that employed a custom script to interface with a laptop computer. The first version of the code
that connected the Balance Board with the PC was written by Nedim Jackman as a part of his undergraduate degree
in Computer Science. Jackman originally wrote the software to "measure the deterioration of aged people's balance
(Schwarts 2009 in conversation with the authors)." Jacob Schwarts, a Masters student studying with author 1,
worked with Jackman to adapt the code so that it translates motion on the WiiBoard to a set of configurable
keyboard signals. See "Jackman" in the references for a link to the code.
Both Schwarts and the authors have used the first version of the code to control the movements of avatars within
UT3; Schwarts to design and demonstrate his graduation project and the authors to create a three dimensional real-
time map of a person's physical movement within a virtual environment. In the first iteration the test subject's
movement is very limited; leaning forward/backwards/left/right replaces taking actual steps. While this represents an
alternative way to interact with the computer (i.e. not a traditional keyboard or game pad, Jefery Shaw's Legible City
1989-1991 is probably the most well know early example of this) it doesn't capture the act of walking passively.
Subsequently the authors worked with Jackman to enable the connection of up to 7 balance boards with the PC. A
video clip (www.russelllowe.com/publications/convr2009/convr2009.htm ) shows author 1 walking forward across
three boards with his movement being translated to the NPA in real time. The second part of the clip shows the
author (and NPA) walking to the left. These clips demonstrate that the steps by the author in the physical
environment produce a related amount of steps by the NPA in the virtual environment. Demonstrations on
www.youtube.com show a Nintendo Wii controller (youtube, 2009) being used to interface with Half-Life 2 (a first
person shooter game in many ways similar to UT3) and implicate a further related area for investigation i.e. the
134
9
th

integration of multiple gyroscopic devices. While these devices do not represent the ultimate solution they do
suggest computer game peripherals could play a role in it.
In contrast to GPS based systems that represent a 'collaboration' between sensors mounted in the environment and
sensors carried by pedestrians INS systems record the pedestrian movements independently. Rizos et al (2008)
present the notion of "bridging GPS gaps" with INS systems which overcomes a significant drawback with INS
systems; "sensor errors that grow unbounded with time". When a player uses the Wii controller to manipulate an
avatar in Half-Life 2 cumulative error is overcome by constant adjustments made by the player in response to their
avatars position compared to their desired position. In the case of the NPA no such adjustments can/are be made (the
pedestrian sees neither the avatar nor environment). The construction site, by very definition, is in constant flux
the physical environment may not exist to support sensors at one stage of construction and then may make GPS
based systems ineffective at a subsequent stage. Strategies for incorporating sensors within buildings as they are
constructed are necessary.
7. CONCLUSION AND FUTURE WORK.
Richard Goodwin's Porosity Project contributed to urban mapping in two very important ways; it recognised that
public spaces don't end at the envelope of a building and by extension it understood that navigating the city is a 3
dimensional proposition. A key factor of a buildings Porosity is the amount of time a person can spend in different
parts of a building; and this duration changes over time.
This research finds that it is possible to represent Porosity in real-time and that an advantageous medium to use to
achieve this is computer gaming technology. Extending from Sun et al the Porosity Lenses add persistent traces of
movement to the "set of architectural clues in sight" that analysts would be able to use to understand pedestrian
movement and space usage within an urban environment. By using computer games the analysts can take advantage
of the "positive benefits of video game play" that include "spatial visualization and mental rotation" (Rosser et al,
2007). The computer game UT3 was chosen to construct these prototypes because it represents one of the latest
generation of computer gaming technologies, it has a comprehensive and interconnected toolset, and the support of a
large game modding community. The Porosity Lenses develop additive and subtractive strategies that have grown
out of an examination and criticism of recent urban mapping projects. In this examination three major issues arise;
the qualitative vs quantitative advantages of utilising Colour Intensity and Size as mechanisms to represent duration
of stay in any one place; the obscuring of data by either subsequent entries by the same person or by entries from
another person; and the representation of the "activity workspace" (Mallasi, 2004). Both mechanisms have strengths
and weaknesses and further work is required to create a hybrid or develop new alternatives. By utilising a base unit
of workspace indexed to the construction worker, which follows them and build's (or carves as the case may be) a
total model of their space use over time, a more fine grained understanding of the activity workspace would result.
To represent Porosity in real-time first one must collect the data in real-time. Allen et al note the "widespread
availability" of low-cost computer game peripherals and with this in mind the authors the authors sought to extend
the modified off the shelf software approach to include hardware. With Nedim Jackman the authors connected 3
Nintendo Wii balance boards to a PC and were able to passively control an Avatar (now a Non Player Avatar, or
NPA). This demonstrates proof of concept, but it is by no means a complete solution; future work will look at
repurposing the Wiimote, Cellphones and Wireless Motes. Finally, while Gronenda et al saw that a virtual
environment would normally be "limited to the size of the user's real environment", and developed "Motion
Compression" to circumvent those limits, the authors see restricting the player to their real environment as an
opportunity to map the limits of that environment by tracing pedestrian movements through it.
8. REFERENCES.
Allen, D. Playfer, J. Aly, N. Duffry, P. Heald, A. Smith, S. And Halliday, D. (2007) "On the Use of Low-Cost
Computer Peripherals for the Assessment of Motor Dysfunction in Parkinson's Disease - Quantification of
Bradykinesia Using Target Tracking Tasks." In IEEE Transactions on Neural Systems and Rehabilitation
Engineering, Vol. 15, No 2. June 2007.
Author 1 (2008) reference regarding large Architecture firms using UT3.
135
9
th

Bouchlaghem, N, Khosowshahi, K. White, J (2000) "Virtual reality as a visualisation tool: benefits and constraints."
CIDAC Special Issue on Visualisation in Architecture, Engineering and Construction. Volume 2, Issue 4.
EPIC Games, Mod Community page: http://udn.epicgames.com/Three/UT3ModHome.html , recovered 14 January
2009
Autodesk press release: http://www.epicgames.com/press_releases/autodesk.html
HKS press release: http://www.hksinc.com/news/2007_10_HKS_Licenses_Unreal.htm
Gemeinboeck, P, Blach, R (2005) "Interfacing the Real and the Virtual: User Embodiment in Immersive Dynamic
Virtual Spaces." In Learning from the Past a Foundation for the Future [Special publication of papers
presented at the CAAD futures 2005 conference held at the Vienna University of Technology / ISBN 3-
85437-276-0], Vienna (Austria) 20-22 June 2005, pp. 171-180
Goodwin, R (2003-2005) Australian Research Council Discovery Grant, Project ID: DP0346062.
http://www.arc.gov.au/
Goodwin, R, McGillick, P, Helsel, S., Tawa, M., Benjamin, A., Wilson, G., (2006) Richard Goodwin: Performance
to Porosity, Craftsman House, an imprint of Thames and Hudson, Australia
Groenda, H (et al) 2005, Telepresence Techniques for Controlloing Avatar Motion in First Person Games, Maybury,
M. et al (Ed's.) INTETAIN 2005, Springer-Verlag Berlin. LNAI 3814, pp44-53
Hansel and Gretel. Fable. http://en.wikipedia.org/wiki/Hansel_and_Gretel recovered 22 January 2009.
Jackman, N (2009) http://code.google.com/p/wiiboard-simple/downloads/list recovered 12 January 2009.
Lara Calder Architects, Sydney: http://www.laracalderarchitect.com.au/
Manovich, L (2000) "The Language of New Media". MIT Press, Cambridge, MA
Microsoft Research, 2005, Computer Gaming to Enhance Computer Science Curriculum.
http://research.microsoft.com/ur/us/gaming/ComputerGamingToEnhanceCSCurriculum.doc recovered
November 2008.
Nintendo: Wii Balance Board http://www.nintendo.com.au/wii recovered 4 February 2009.
Nold, C (2004 - Ongoing) http://www.biomapping.net/ recovered November 2009.
Petersen, T (2005) "Generating Art from a Computer Game. An Interview with Alison Mealey"
http://www.artificial.dk/articles/alison.htm recovered 13 July 2009.
Polak, E (2002) "Amsterdam RealTime: Diary in Traces" For the exhibition "Maps of Amsterdam 1866-2000 at the
Amsterdam City Archive. http://project.waag.org/realtime/en_frame.html recovered 9 January 2009.
Price, C (2006) "A Crisis in Physics Education: Games to the Rescue!" in ITALICS, Innovation in Teaching And
Learning in Information and Computer Sciences, Volume 5 Issue 3.
http://www.ics.heacademy.ac.uk/italics/vol5iss3/price.pdf recovered 1 Feb 2009.

Reades, J, Calabrese, F, Sevtsuk, A, Ratti, C, (2007) Cellular Census: Explorations in Urban Data Collection, IEEE
Computer Society, Pervasive Computing, Vol. 6, No 3, July-September.
Rizos, C, Grejner-Brzezinska, D.A, Toth, C.K., Demster, A.G, LI, Y, Politi, N, and Barnes, J. (2008). A hybrid
system for navigation in GPS-challenged environments: Case study. 21st Int. Tech. Meeting of the Satellite
Division of the U.S. Inst. of Navigation, Savannah, Georgia, 16-19 September, 1418-1428.
Rosser, J, Lynch, P, Cuddihy, L, Gentile, D, Klonsky, J and Merrell, R (2007) "The impact of Video Games on
Training Surgeons in the 21st Century. Archives of Surgery, Vol. 142 No. 2, http://archsurg.ama-
assn.org/cgi/content/full/142/2/181 recovered 13 January 2009.
Shaw, J (1989-1991) "Legible City" http://www.jeffrey-shaw.net/html_main/frameset-works.php3 recovered 14 July
2009.
136
9
th

Sun, C, De Vries, B, Dijkstra, J, (2007) Measuring Human Behaviour Using a Head-Cave. CAADFutures, pp. 501-
511.
Thompson, C (2007) Halo 3: How Microsoft Labs Invented a new Science of Play, Wired Magazine, issue 15.09
available online here: http://www.wired.com/gaming/virtualworlds/magazine/15-09/ff_halo recovered 13
July 2009.
Townsend, A (2000), Life in the Real-Time City: Mobile Telephones and Urban Metabolism. Journal of Urban
Technology, 7:2, 2000, pp85-104
YouTube (2009) Search "WiiMote playing Half-Life 2 on a computer".
http://au.youtube.com/watch?v=asY_I8y6C0M recovered 16 January 2009.
137
9
th

VIRTUAL REALITY USER INTERFACES FOR THE EFFECTIVE
EXPLORATION AND PRESENTATION OF ARCHAEOLOGICAL SITES
Daniel Keymer, Mr,
Department of Computer Science, University of Auckland;
dkey012@aucklanduni.ac.nz
Burkhard Wnsche, Dr,
burkhard@cs.auckland.ac.nz
Robert Amor, Associate Professor,
trebor@cs.auckland.ac.nz
ABSTRACT: Archaeological virtual environments are computerised simulations allowing the study and exploration
of archaeological sites. For architecture students and researchers at the University of Auckland they provide
several advantages compared to traditional methods of study and exploration such as site visits, illustrations and
books. Advantages include that there is no physical travel required, greater amounts of information can be provided
in a more accessible manner than with maps or diagrams, and different representations of the site can be created,
e.g., before modifications and expansions. The sites that archaeological virtual environments represent can contain
many structures and thousands of artefacts distributed over a wide area. As a result users find it hard to get an
overview of the site or to focus on particular aspects. Furthermore data on these sites is often gathered over a long
period of time using different processes and media, which makes it difficult to present effectively to a student body.
In this project we present solutions to these problems tailored to the needs of different user groups such as
archaeologists, architects and architecture students. The requirements of different user groups were analysed and
Virtual Reality technologies were developed to facilitate the exploration of archaeological sites, in order to retrieve
information effectively and to gain new insight into the site and its inhabitants. These technologies are demonstrated
within a new reusable archaeological virtual environment framework, which is used to create virtual environments
for archaeological sites. The framework is built upon a game engine, resulting in a quicker development cycle and
more realistic rendering than would be feasible if it were developed from the ground up. In contrast to previous
applications our framework enables the integration of a wide variety of media. This dramatically facilitates content
creation, which is usually very time consuming, expensive, and requires skilled modellers and/or animators. Our
framework provides simple interfaces to create a 3D context (terrain and simple models) and then integrates more
easily obtainable representations such as images and movies for providing visual details.
The technologies designed and implemented included the integration of QuickTime VR into a game engine, which
allows a commonly-used medium for recording scenes to be used within a virtual environment. The two media
integrate well and, while not seamless, the new representation enables a focus-and-context style exploration of the
domain. We also present a data model for archaeological sites that supports a wide variety of information types
including multimedia. It is independent of the rendering engine used, allowing archaeological virtual environments
to be extended and upgraded more easily. Using these representations we investigated new metaphors for
navigating and interacting with archaeological virtual environments, including interactive maps, guided tours,
searching mechanisms, time-lines and time-of-day settings for controlling sunlight direction.
The techniques afford users a richer, more informative experience of archaeological virtual environments. They can
be adapted to a broad range of archaeological sites, even where data was gathered by multiple differing methods.
KEYWORDS: Virtual environments, novel interaction interfaces, game engines, archaeology, data representations
139
9
th

1. INTRODUCTION & BACKGROUND
Three-dimensional virtual environments have been investigated since the 1990s for use in the study of archaeology.
Their uses range from purely research-oriented tools to aid in the visualisation of archaeological data through to
explorable digital renditions of full archaeological sites for uses such as education and virtual tourism. Whichever
the application, virtual environments provide a number of advantages:
They allow people to visit sites without having to physically travel to them.
They improve the ability of the observer to visualise and understand a site compared with traditional
media such as photographs and maps.
They provide a more interesting and immersive view to the casual observer than such traditional
media.
They allow a wider range of viewpoints from which to study a site, including in a temporal sense; for
example, an observer can watch a site develop over time
They can present additional information on features of interest in a coherent, easily accessible manner;
for instance, an observer may examine details of an archaeological artefact presented in the virtual
environment without having to refer to a separate text. This is especially useful if the description
refers to the objects location and its relationship the environment and other objects.
They allow users to interact with the environment, e.g. by moving, adding and removing objects and
structures or by changing scene properties such as lights and materials. This can help understanding
the motivations for an existing architectural design and its changes over time.
They allow people in separate geographical locations to work collaboratively on a single site.
They can easily be distributed to the general public.
Archaeological virtual environments may be roughly divided into two classes; research aids and historical
reconstructions. The former represent the site as it exists today and provide the means to explore and study it; the
latter attempt to recreate the site as it was at one or more points in history. Both types of environments exist,
although historical reconstructions have been more widely publicised.
Virtual environments can be greatly beneficial for archaeological research. Before the advent of computer
technology, visualisations of archaeological data were created in the form of two-dimensional diagrams or
illustrations (Reilly and Rahtz 1992). The immense quantities of data generated from archaeological excavations and
surveys are often very difficult to adequately display in these formats, however. For example, stratigraphy the
representation of the layers of earth on a site and the objects and soil types contained within is represented using
two-dimensional diagrams called sections (Renfrew and Bahn 2004). This data is inherently three-dimensional
(Barcel et al 2000), and would be better presented using three-dimensional visualisation methods. Two-
dimensional methods also struggle to adequately represent the associations between finds on a site because of the
limited information that can be displayed (Barcel et al 2000). Interactive three-dimensional visualisations such as
virtual environments greatly ease this restriction, and provide an increased number of ways to visualise and work
with data. For example, neighbouring objects can be computationally analysed to determine whether they might
actually be fragments of a single object. Examples of these kinds of archaeological virtual environments include
ARCHAVE (Acevedo et al 2001), which allows the analysis of stratigraphy and the finds within an archaeological
site, and GeoSCAPE (Lee et al 2001), which facilitates the recording of the position of the features of an
archaeological site including artefacts, structures and other finds, and their later visualisation.
Virtual environments depicting historical reconstructions have often been aimed towards the general public, but they
can still be helpful to archaeologists. They may, for instance, be used to formulate and evaluate hypotheses about
questions such as how the site was used. They also allow a broader variety of ways to present information more
richly than earlier technologies: any number of viewpoints is possible in a virtual environment, features of the site
such as buildings and artefacts may be hyperlinked to relevant information such as bibliographic resources, and
querying can be used to locate and concentrate on important details (Renfrew et al 1996 and Barcel et al 2000).
One of the strongest advantages of these kinds of virtual environments is their usefulness for the dissemination of
research: archaeological research is normally published in a printed journal, and all too often a long time passes
140
9
th

between the actual fieldwork and analysis and its publication (Barcel et al 2000 and Renfrew and Bahn 2004).
Virtual environments can be distributed via electronic means such as the Internet. Furthermore, they can easily be
updated as new information emerges, increasing the frequency of the publication of this information as well as
reducing the delay. Virtual environments have the additional advantage of being easier to interpret than a series of
disparate diagrams and explanatory text, improving their accessibility to both academics and laypeople.
The use of virtual environments for virtual museums or tours aimed at the general public is also very common
(e.g. 3D Rewind Rome (2009), and the work of Gaitatzes et al (2001) and Kim et al (2006)). These are gradually
advancing in complexity and capabilities, with continuously improving graphics and some featuring sophisticated
animations as well.
Significant challenges are still faced in the development of virtual environments to represent archaeological sites,
especially as the scope and detail of the information presented increases. It is necessary to capture and digitize 3D
data before it can be used in a virtual environment, and it is difficult to achieve this cost-effectively while
maintaining high visual fidelity. Furthermore, information-rich virtual environments must merge data collected from
often disparate sources such as photographs, 3D models, GIS data and reference texts into a cohesive whole. This
data can be incomplete, tends to contain uncertainties, and may change over time as more information is collected.
The large quantity and complexity of the data available for an archaeological site necessitates careful consideration
of the representation of that data and how it is accessed. Our project has investigated methods to allow
archaeological virtual environments to present this data in a coherent, informative and easy to use manner, while
remaining cost-effective to create and maintain.
2. SELINUS CASE STUDY
During the development of our project we created a virtual environment of the ancient city of Selinus a Greek
colony in Sicily as a case study. The University of Aucklands School of Architecture and Planning, in
conjunction with its counterpart in the Universit degli Studi di Palermo (University of Palermo), runs a joint
program every few years in which students from Auckland and Palermo visit archaeological sites including Selinus,
and work together on architectural projects (Milojevic 2007).

FIG. 1: Overview of the Selinus site.
A multimedia DVD containing photographs, maps, 3D models, QuickTime VR panoramas, and textual resources
has been created to give these students an introduction to the Selinus site, its contents, and its history. However, it is
not as effective or as immersive as a virtual environment would be, and thus its authors wished to create such an
environment to improve the quality of their learning resources. We have used this project as an example to help
guide our decisions in the design of a virtual environment creation system that can be used for a broad variety of
purposes, which may not even include archaeological research.
141
9
th

3. REQUIREMENTS
In order to design a broadly-applicable virtual environment system we needed to consider the needs of the wide
variety of users it would have. These included:
creators of virtual environments, such as researchers and other academics from the archaeological and
architectural fields;
users from the architectural field who would be using the system to explore archaeological sites; and
users from the archaeological field who would be using the system to study an archaeological site.
Users wanting to create virtual environments of a real location may have varying levels of computer skills, and are
often not programmers. Content creation tools such as level editors of game engines and modelling and animation
applications have made this task easier, but do require some artistic skills, reasonable technology skills (e.g. use of
scripting languages), and are time consuming and expensive. Most large-scale Virtual Reality application we are
aware of use professional designers and animators. A suitable application for laymen is Second Life, which,
however, does not have any tools for creating archaeological content. In order to make content creation more
efficient and effective we must utilise pre-existing data such as diagrams, 3D models, contour maps, and the like.
We surveyed users of archaeological environments and evaluated the literature and found (Keymer 2009) that users
of virtual environments from the architectural field tend to be most interested in the aspects of a site that could be
considered more artistic; its spatial features, its form, its appearance and so on. They are particularly interested in
the structures on a site, and thus are often responsible for creating historical reconstructions or visualisations of an
archaeological site or ancient building. In order to appreciate the form and shape of a site, they prefer to be able to
view them from as many vantage points as possible. Views from the ground are important, as they reflect the site
from the perspective a common person may have seen it; however views from the air can also be particularly
informative. Human activity is important to architects; this is one of the main reasons that the archaeological
artefacts on a site are interesting to them. The most important aspects of these artefacts are, again, largely artistic;
their minor details are less important than their presence, spatial attributes such as their arrangement, and what they
indicate about the human presence in the area. Thus architects may tend to browse through areas rather than study
particular groups of artefacts in detail.
Users from the archaeological field, by contrast, study sites from an anthropological perspective; they seek to
understand the people that lived there, not just the artefacts, buildings and other features that make up the site
(Renfrew and Bahn 2004). For this reason the crucial information from an archaeologists perspective is the cultural
significance of an artefact or a structure for example who used it, how they used it, and what it indicates about the
society that created it. How this information should be presented depends on whether the virtual environment is
being used for presenting past findings or for original research; if the former is true an interpretation of the
significance of an object can be provided directly, but if the latter is the case as much information as possible should
be presented about each feature of the archaeological site to maximise the chances that a researcher will be able to
draw useful conclusions (Renfrew et al 1996).
These requirements can often conflict with each other; an environment that is useful for an archaeologist is likely to
contain a great deal of information which would be unnecessary or even distracting to an architect. Therefore a
virtual environment creation system that is useful to both would need to be able to tailor the resulting virtual
environment to present the level of information required.
4. PROJECT DESIGN - GAME ENGINE
The rendering framework was the first stage of the development of the project as it would form the foundation for
later work. We elected to build it using a pre-existing framework or technology as a base. There were three main
reasons for this. Firstly, the development of a new framework would not address any of the goals of this project; it
focuses on user interface techniques, and features such as a custom rendering engine were considered to be unlikely
to add significant value to these. Secondly, the use of a pre-existing framework would accelerate the development of
the system overall, and the features that would add value in particular. Thirdly, the disadvantages and limitations of
such an approach are minor; although it may limit the flexibility of the overall system somewhat, this can be largely
mitigated with careful choice of a technology appropriate to the project.
142
9
th

We analysed a variety of options when choosing a basis for our virtual environment rendering framework, including
general-purpose 3D visualisation technologies, previously developed archaeological tools, and game engines
(Keymer 2009). Game engines fulfilled our requirements best since they are cheap, are optimised for consumer level
hardware, are frequently updated to use the latest graphics technologies, provide intuitive interaction tools, and have
a large user base resulting in well tested and stable code. We eventually settled on Esperient Creator (2009), as we
received access to the SDK, had previous experience with it, and because it is highly flexible, had excellent tools,
strong plug-in support, multi-player support (for collaborative visualizations), and is targeted at the development of
non-game virtual environments such as those we wish to support.
Even with a well-suited engine, however, we still needed to create our own pipeline for creating, rendering and
exploring archaeological virtual environments. We achieved this through the use of two major components: one of
these is a stand-alone program to collect various types of archaeological data, compiling them into a computerised
description of the archaeological site. The other component is a plug-in for the Esperient Creator engine which
interprets this description and uses it to display and facilitate interaction with the virtual archaeological site.
This plug-in is designed to be re-usable and extensible, in order to support a wide variety of kinds of archaeological
sites. It has a two-layer, modular architecture which separates the concerns of the data model from the engine used
to represent it making the data easier to work with and more portable and separates distinct blocks of
functionality into their own modules, which helps to make new functionality easier to introduce.

FIG. 2: Virtual environment creation interface: terrain loading and modification.
4.1 Data Model
Archaeological data sets can be extremely large, diverse and feature rich. In addition some data, such as building
structures, can change over time. We have created a custom data model which supports dynamic scene generation,
efficient data access and manipulation, and interaction with other systems or APIs. The data is imported from an
XML scene descriptor file, which facilitates interchange of data, extension of the model, and integration with web
interfaces. The data model is divided into the following components which have different attributes and functions:
Environmental components (terrain, seas, rivers, vegetation)
143
9
th

Man-made structures (buildings and other structures such as walls and moats)
Archaeological artefacts
Multimedia components (e.g. photographic images of the site and objects, including QuickTime VR
panoramas and object movies.)
Temporal entities (eras)
All of these components can be interrelated. For example, man-made structures and artefacts can be associated with
one or several eras and multimedia components can be associated with positions in the terrain or artefacts. Each of
these data entities contains attributes describing it, such as size, shape, age, date created, date found, bibliographic
references, multimedia descriptions (photos, object movies tec.), and relationships with other elements of the
archaeological site. More details of the data model and attributes are found in (Keymer 2009).
Different data entities are generated and loaded using different tools using common interfaces. For example, figure 2
illustrates the terrain generation using a digital elevation map (DEM) representation where height values correspond
to gray scale values of an image. We have also developed an application for creating DEMs by drawing and semi-
automatically labelling contour lines or importing them from GIS applications (Xie and Wnsche 2009).

FIG. 3: QuickTime VR within the game environment.
4.2 QuickTime VR support
Supporting some types of data used to create archaeological sites can be more difficult than others; one of the most
difficult was QuickTime VR panoramas. These allow a user to view a 3D scene from a single point by wrapping a
144
9
th

photograph or series of photographs onto a virtual cylinder or sphere. The user can click and drag with the mouse to
rotate their view of the scene. QuickTime VR panoramas are widely used by a number of archaeological projects,
including the Selinus project, because they can be produced relatively easily and inexpensively, and allow an
immersive view of a site that is otherwise difficult to obtain.
We allow the user to transition from a view of the virtual environment to a view of the QuickTime VR panorama,
retaining the same control system from the environment, which is also drawn around the border of the QuickTime
VR panorama to provide a frame of reference. The QuickTime VR panorama provides an interactive feature rich
representation without expensive and time-consuming creation of 3D content. Note that 3D content can be created
automatically from video images, but this is computationally expensive, heavily influenced by environmental
parameters (illumination, shadows), and requires expensive well calibrated hardware. In addition the resulting
scenes usually contain many small errors which severely reduce viewing pleasure.
This process was challenging to implement; QuickTime is normally designed to be used by embedding a player in
an appropriate place inside an application window, but due to issues in timing the rendering of the player with the
in-built rendering cycle of Esperient Creator, we had to introduce an extra stage of indirection into the process by
rendering the player to a non-visible section of memory, the contents of which were then copied into an Esperient
texture buffer. Details are given in (Keymer 2009). While this process is more complicated and less efficient than
drawing directly to the screen, it does allow us more flexibility with the use of the QuickTime player's output. The
results are illustrated in figure 3. The view direction inside the virtual environments changes with the view of the
QuickTime VR movie such they are always aligned. Small offsets are unavoidable due to inaccuracies in the
modelled scene and lack of information about the exact camera position for the QuickTime VR movie.
4.3 Navigation
Another important concern is how users move through and navigate the virtual environment. Architects as we
discovered are particularly interested in being able to view a site from the ground, but also find it helpful to view it
from the air. Thus our system had to handle both, allowing the user to switch between them quickly and easily.
Ground-based movement had the higher priority of the two.
The size of large sites such as Selinus also has impacts on navigation. It will take some time to cross a large site
if the user can only move slowly; however if movement is too fast the system will be less usable and less immersive,
and disrupt the users perception of the scale of the site. The system had to allow the user to move at a normal
walking speed, but also provided methods for crossing large distances that do not disrupt perception of scale or
immersion. The portal concept from computer games is a suitable method to jump quickly between logically
connected positions, e.g. buildings which related functions spread over a wide area. The Strider concept, which we
introduced in (van der Linden et al. 2004), allows exploration with smoothly varying speed, context and perspective.
A major difference to games and other virtual environments is that archaeological sites change over time. The user
should hence not only be able to explore a site in the spatial domain, but also in the temporal domain. The time
period displayed in the virtual environment is controlled by a timeline slider as illustrated in figure 4. The slider is
manipulated using the mouse, and the adjacent text provides a visual indication of the currently selected era. Only
objects valid during that era are rendered. In order to achieve this each data model entity presently artefacts and
buildings that is affected by the timeline system contains a list of the eras it is present in and absent from. Terrain
is less likely to change over time, but the system could be extended to support terrain if such changes are later found
to have been significant.
Guided navigation methods such as tours allow users to become acquainted with a site quickly. While non-
essential we believe they would be a valuable addition to our system. Tours should utilise both the spatial and
temporal domain and should be easy to find, easy to use, and effective at improving understanding of the site.
The system should also support a mechanism that allows artefacts and other important elements of an archaeological
site to be found quickly. This would be especially important for archaeologists to allow them to locate related
finds, for instance but it would also be helpful for anyone who is particularly interested in one location or artefact
on the site. This kind of search mechanism would therefore need to be designed to identify individual entities
based on a variety of commonly-required constraints, whether by individual object attributes, groups of attributes, or
relationships to other objects on the site.
145
9
th

FIG. 4: Timeline control to filter site view.
4.4 Artefact display
There are a number of key requirements the artefact handling features of the virtual environment system had to
satisfy. These include abilities such as being able to represent the artefacts on screen, conveying as much
information about them especially important details as quickly and concisely (such as in visual form) as possible,
allowing the user to easily interact with them as necessary, and managing situations where artefacts are distributed
quite densely.
We chose to represented artefacts within the virtual environment by small 3D objects, or icons, which indicated
the type of the artefact; for instance an individual amphora may be represented by a generic amphora icon.
Because there are relatively few artefact types, only a few models need to be provided, which reduces the amount of
time and resources necessary to build virtual environments containing large numbers of objects.
When the icon is clicked a detail view is shown which displays more information about the selected artefact. This
view displays material that cannot be easily displayed in the virtual environment such as textual information and
visual media and information that may be represented in the environment but cannot be accurately interpreted
there such as its dimensions and significance.
The detail view consists of two panes as in figure 5. The left pane displays a selection of visual media, while the
right contains exclusively textual information about the artefact. Media in the left pane need not be static images;
QuickTime VR object movies are shown here as well, and the system is constructed to be extended to support other
media types including animations and movies as needed. This is demonstrated by the QuickTime VR integration
which is essentially an advanced movie format.
5. RESULTS, CONCLUSIONS AND FUTURE WORK
We have presented a system supporting the display and exploration of archaeological virtual environments that can
be created from a broad range of types of media. This work was motivated by our observations that much existing
data recorded for archaeological sites is difficult to use in existing virtual environments, and that they could be
improved if these types of data were supported. After analysing the problem and user requirements we identified key
features to incorporate into our system, then developed designs for these features and implemented them into a
working application.
146
9
th

We have evaluated technologies and found that game engines are best suited because using they provide multi-
player capabilities, intuitive interfaces, tools for content creation, and they support common 3D data models. Our
system separates the content of a virtual environment from the system used for the presentation of that environment.
This helps it to achieve the objective of supporting a wide range of archaeological sites. We created a conceptual
meta-model of an archaeological site describing the kinds of features and information that can make up an
archaeological site; this information is used to create collections of data files that can be used by the same
environment. One of the most effective features of this approach is that it allows pre-existing data that has been
collected for an archaeological site to be used for creating a virtual environment. This data may be stored in a
diverse range of formats, such as GIS data, photographs, models and QuickTime VR movies. By using this data as-
is rather than requiring custom-designed content be used for the site, we have created a system that allows virtual
environments to be created more easily, quickly and efficiently. Furthermore it can be updated simply by adding
new content, without requiring an expensive and time-consuming conversion into a 3D representation.

FIG. 5: Object icons in the game environment and the detail view.
Integrating QuickTime VR movies into the rendering of the scene provided rich content with suitable context.
Identifying the precise camera position for a QuickTime VR view is difficult without additional data such as its GPS
position. However, even without seamless integration the media blended well and allows a focus-and-context style
exploration of the domain. The game environment provides an overall context for QuickTime VR movies and
enables users to get an overview of the entire scene and the QuickTime VR movies camera position within it. Vice
versa the QuickTime VR movie is content rich and provides a more realistic representation than the modelled
environment and hence provides a better context for understanding the terrain, e.g. how the architecture blends into
the natural environment.
We also performed a basic usability study and found that navigation is easy, although the existing Selinus virtual
environment is relatively simple. More challenging scenarios are required to test how well the system performs in
complex environments. Interaction with QuickTime VR panoramas works quite well. They are easy to see and
147
9
th

activate, and view rotation is synchronised accurately. Rotating the view with the mouse results in sluggish
handling, however, which needs to be improved. Interaction with artefacts is relatively easy, but they can be difficult
to see at a great distance due to their size. The planned inclusion of a top-down map should alleviate this problem.
The Selinus site takes a long time to move around at present, and this could become frustrating for users. This may
be partly due to performance issues that slow down movement, but in any case it is likely that faster movement, e.g.
by using the previously mentioned Strider interface, will be required.
Overall our results indicate that the developed tool is intuitive to use and provides an easy way to create content rich
multi-media supported 3D representations. More detailed user studies are required to test the effectiveness of the
tool. In particular we want to test whether our tool helps users to explore an environment more efficiently, whether
users memory of scene content and spatial relationships improves, and whether users can solve archaeological
problems more effectively. We also want to do add more capabilities for automating the creation of scene content,
e.g. modelling of 3D objects and structures from images using photometric stereo, integrating GIS data (if
available), and automatically creating guided tours based on spatial, temporal and semantic relationships.
6. REFERENCES
3D Rewind Rome (2009), [cited 25 January 2009]; Available from: http://www.3drewind.com/.
Acevedo, D., Vote, E., Laidlaw, D.H. and Joukowsky, M.S. (2001), Archaeological data visualization in VR:
analysis of lamp finds at the great temple of petra, a case study, Proceedings of Visualization '01, IEEE
Computer Society, San Diego, California, 493-496.
Barcel, J.A., Forte, M. and Sanders, D.H., eds. (2000), Virtual reality in archaeology, BAR international series,
Vol. 843, Oxford, England, 262pp.
Esperient Creator (2009), [cited 15 July 2009]; Available from: http://www.esperient.com/.
Gaitatzes, A., Christopoulos, D. and Roussou, M. (2001), Reviving the past: cultural heritage meets virtual reality,
Proceedings of Virtual reality, archaeology, and cultural heritage, ACM, Glyfada, Greece, 103-110.
Keymer, D.J. (2009), User Interfaces for the Effective Exploration and Presentation of Virtual Archaeological
Sites, ME Thesis, University of Auckland, Auckland, New Zealand.
Kim, Y.-S., Kesavadas, T. and Paley, S.M. (2006), The Virtual Site Museum: A Multi-Purpose, Authoritative, and
Functional Virtual Heritage Resource, Presence: Teleoperators & Virtual Environments, 15(3), 245-261.
Lee, J., Ishii, H., Dunn, B., Su, V. and Ren, S. (2001), GeoSCAPE: designing a reconstructive tool for field
archaeological excavation, CHI '01 extended abstracts on Human factors in computing systems, ACM,
Seattle, Washington.
Milojevic, M. (2007), Exceptional Access: Re-Presenting Ancient Selinus Virtually, Proceedings of Interface:
Virtual Environments in Art, Design and Education, Dublin, Ireland, 6-7 September.
Reilly, P. and Rahtz, S., eds. (1992), Archaeology and the Information Age: A Global Perspective, Routledge,
London, England.
Renfrew, C., Forte, M. and Siliotti, A., eds. (1996), Virtual archaeology: great discoveries brought to life through
virtual reality, Thames and Hudson, London, 294pp.
Renfrew, C. and Bahn, P.G. (2004), Archaeology : theories, methods and practice, Thames & Hudson, 4th ed.,
London, 656pp.
van der Linden, J., Li, J., Lobb, R., Novins, K. and Wnsche, B. (2004), Strider: A Simple and Effective Terrain
Navigation Controller, Proceedings of IVCNZ '04, Akaroa, New Zealand, 21-23 November, 95-100; URL:
http://www.cs.auckland.ac.nz/~burkhard/Publications/IVCNZ04_vanderLindenEtAl.pdf
Xie, X. and Wnsche, B. (2009), Efficient Contour Line Labelling for Terrain Modelling [submitted for
publication]
148
9
th

INTERACTIVE CONSTRUCTION DOCUMENTATION
Antony. Pelosi, Mr,
Massey University;
a.pelosi@massey.ac.nz
ABSTRACT: This paper discuses the potential of moving beyond 2D paper-based construction documentation by
leveraging the power of real-time 3D computer gaming engines to produce first person view digital environments to
explain design intent for the architectural, engineering and construction industries. By comparing methods used in
the aerospace and automobile industries to requirements of the AEC industries, this paper outlines the benefits of
interactive 3D construction documentation.
KEYWORDS: Architectural hyper-model, computer game engine, real-time 3D, interactive, construction
documentation.
1. INTRODUCTION
There are inefficiencies in the production and editing of working drawings. Building information modeling (BIM)
has started to reduce these inefficiencies, but it remains focused on 2D paper based drawings as the final output,
subsequently losing the benefits of an intelligent 3D model. This paper will examine how to leverage BIM and 3D
computer gaming engine technology to form an architectural hyper-model that would be a valuable supplement to
the conventional scaled 2D construction drawing documentation found on construction sites. Professor Chuck
Eastman et-al states that, Three-dimensional models and 4D simulations produced from building information
models are far more communicative and informative to lay people than technical drawings (Eastman et al., 2008).
With the increasing availability of powerful real-time 3D digital environments, like Google Earth and computer
games, the next generation of people in the building industry are already comfortable with interacting and
communicating within real-time 3D digital environments. The Construction 2020 report prepared by the
Cooperative Research Centre (CRC) for Construction Innovation outlines visions and suggests goals for the
construction industry. They state for communication and data transfer to be seamless, enabling transfer without
interruption and delay, and include mobile devices providing a commercially secure environment. These
technologies will be embedded within both construction products and processes to improve efficiency and
effectiveness. The knowledge economy will require property and construction to become more engaged in IT
developments.
2. CONSTRUCTION VISUALIZATION
Historically, architects have produced a set of 2D drawings and specifications that have abstracted information
spread across sheets of paper at different scales and points of view. Coupled with the fact that a set of drawings is
typically multiple sheets, the proposed building project is made difficult to visualize as a complete object as well as
the narrative they describe for the process of construction. Contractors have to piece together information from 2D
drawings; this form of communicating has been the established practice for over 500 years. The methods of
architectural design visualization have undergone major changes over the past 50 years with the introduction of the
computer and CAD software in to architectural practice, but we have seen very little improvement or innovation to
construction documentation over the same period.
The current shift in the AEC industries is to building information modelling (BIM). Professor Chuck Eastman et al,
define BIM in their book BIM Handbook as a modelling technology and associated set of processes to produce,
communicate, and analyze building models.
Building components that are represented with intelligent digital representations (objects) that 'know
what they are, and can be associated with computable graphic and data attributes and parametric rules.
Components that include data that describe how they behave, as needed for analyses and work
processes, e.g., takeoff, specification, and energy analysis.
149
9
th

Consistent and non-redundant data such that changes to component data are represented in all views of
the component.
Coordinated data such that all views of a model are represented in a coordinated way.
They then go on to quote Mortenson Company, a construction contracting firms definition of BIM technology as,
an intelligent simulation of architecture. To enable Mortenson to achieve integrated delivery, this simulation must
exhibit six key characteristics:
Digital;
Spatial (3D);
Measurable (quantifiable, dimension-able, and query-able);
Comprehensive (encapsulating and communicating design intent, building performance, constructability,
and include sequential and nancial aspects of means and methods);
Accessible (to the entire AEC/ owner team through an interoperable and intuitive interface); and
Durable (usable through all phases of a facilitys life).
The use of BIM in practice has many well-documented benefits for the AEC industries (Eastman et al., 2008, Issa
and Suermann, 2007) The majority of these benefits focus on the design and pre-construction phase of building and
not the transfer of information to parties new to a project or who are only required to deal with a small element, sub-
contractors for example.
Professor S. N. Pollalis from Harvard Design School comments that Computers today are used to produce 2D
drawings faster and, as a result, in large quantities. However, one should question the basics and re-deploy
computers to solve the problem of over-documentation as opposed to using them for multiplying the existing spill-
out of information in a paper tilted Understanding Changes in Architectural Practice, Documentation Processes,
Professional Relationships, and Risk Management. (2006)
Research from Center for Integrated Facility Engineering (CIFE) at Stanford and the National Institute of Standards
and Technology (NIST) show how traditional practices are inefficient and error prone, contributing to budget
overruns. The study by CIFE illustrated that productivity in the construction industry has remained constant over the
last 40 years while just about every other industry has seen productivity growth.
The results from a study carried out by NIST show that inefficient interoperability accounted for an increase in
construction costs of over US$6 per square foot for new construction and an increase of US$0.23 per square foot for
operations and maintenance, resulting in a total added cost of US$15.8 billion [in America]. (Gallaher et al., 2004)
In the context of the NIST study, one of the key characteristics of inadequate interoperability is mitigation of digital
and/or paper files that had to be manually re-entered into multiple systems and request for information management.
One of the benefits of the data re-entry is the increased spatial understanding of the proposed building for the people
entering the data, however this knowledge is not past on. Considering this fact, how can this increased spatial
understanding be delivered H without costly data re-entry and be accessible to all parties involved in the life of a
building? After evaluating methods used in other industries and comparing the different requirements, this paper
proposes that the answer lies with the use of real-time 3D interactive digital environments.
The use and accompanying benefits of 3D models and BIM have largely been limited to design efficiencies, conflict
detection, quality improvements and on time completion (Issa and Suermann, 2007), with the final output still
emulating 2D hand drawn documents. BIM technology is currently focused on the creation of the model, not the
accessibility of the model, only providing an increased number and complexity of drawings and specifications to the
construction contract documents. (Pollalis, 2006).
The role of paper-based 2D working drawings is set to change over the next ten years. It is still unclear what these
changes will be and the impact on the AEC industries. BIM will be influencing these transformations. One of the
roles of drawings in the construction industry is as contractual documents; professor Chuck Eastman et al state,
there are indications that BIM models can better serve this purpose, partly because of their improved accessibility
to non-professionals (2008). The question remains how do you communicate design and construction intent to the
people building a building? Many architects already provide axonometric and perspective drawings as part of the
150
9
th

drawing set to help explain design intent, with positive feedback from contractors. Current drawing standards have
evolved due to the technological limitations of paper and the tools used to draw; orthographic projects were
developed to enable measuring of distances on paper. If we are to move beyond a paper based delivery system, then
the current drawing symbols and formatting conventions will need to be replaced to reflect the new methods.
3. DIGITAL DOCUMENTATION PROCESSES
This paper reviews digital documentation processes used in the aerospace and automobile industries, gaining an
understanding of their workflows and outputs in relation to the construction industry; the new Boeing 787
Dreamliner has completely paperless documentation, for example.
The aerospace and automobile industries have been leading the implementation of CAD, solid modelling, product
life management (PLM) and digital documentation, providing huge productivity gains across the entire life of
production, from design through to maintenance. The study by NIST mentioned in section 2 notes that
Computer, automobile, and aircraft manufacturers have taken the lead in improving the integration of design and
manufacturing, harnessing automation technology, and using electronic standards to replace paper for many types of
documents. Unfortunately, the construction industry has not yet used information technologies as effectively to
integrate its design, construction, and operational processes. There is still widespread use of paper as a medium to
capture and exchange information and data among project participants. (Gallaher et al., 2004)
There has been a shift from 2D line based graphic documentation to real-time 3D interactive models for manufacture
and maintenance within the aerospace and automotive industries. They are developed directly from the design CAD
3D model, resulting in reduced production times for 3D and 4D graphic documentation and huge cost savings with
faster time to market. Bell Helicopters have updated their graphical documentation processes, with the technology
shift saving 39,000 labor hours in development of technical publications for one aircraft alone. Thats an 80-
percent savings in the time to produce graphics of much higher quality, and that are much more portable, resulting
in netted savings in another project of 26,000 labor hours a year.
Studies have shown that people are more likely to retain information by doing versus only reading (Dale, 1969).
Interactive 2D, 3D and 4D training contents simulate doing. Virtual simulations, and virtual reality are a lower cost
alternative to actual product mock-ups, physical simulations, and other non-electronic learning by doing
approaches. A white paper published by visual product communication and collaboration company, Right
Hemisphere, states that Interactive training has become mission critical in aerospace, defence, automotive, and
other industries (Right Hemisphere, 2005)
To achieve fully digital documentation companies like Boeing, Bell Helicopters and Daimler Chrysler are using
programs like Right Hemispheres Deep Exploration with Adobe 3D pdfs and real-time 3D authoring programs
such as Esperient Creator to leverage complex CAD data for manufacture and maintenance training documentation,
thus providing non-paper based communication of complex procedures that have historically been expensive and
complex to produce and distribute globally. The digital multidimensional documentation has reduced time to market
and costs by providing a platform that offers a secure, clear image based multi-lingual instruction package. This is
all achieved with great interoperability, without the need to re-input the core design information, reducing expensive
authoring time once design is complete.
The automotive and aerospace industries have large lead times for design with hundreds or thousands of human
hours working on one project that will be mass-produced in the hundreds and thousands or even millions, for global
distribution and consumption. The resulting design CAD model is an exact dimensional correct representation of the
finished product, compared to a building project that is usually a single output of a design solution dealing with
localized climatic conditions, building code requirements, and different client demands and expectations. The design
CAD model has a dimensional different condition to the building, usually based on a grid system that controls the
areas of tolerance.
CAD packages developed for the automotive and aerospace industries have been used in the AEC industries with
outputs that would not have been possible with standard AEC software. Frank O, Gehrys Guggenheim Museum,
Bilbao is one of the most notable examples. Dassault Systems CATIA, the world's most widely used parametric
modelling platform for large systems in aerospace and automotive industries (Eastman et al., 2008) is the backbone
for Gehry Technologies Digital Project, BIM software tailored to the building industry.
151
9
th

There has been very little change in the type of documentation of design intent in the AEC industries since the
introduction of drawing software into the workflow. Mentioned in section 2 there has been an increase in
documentation due to a change in relationship between architect and contractor, and a raise in litigation risk.
4. ARCHITECTURAL HYPER-MODEL
How can the AEC industry learn from the techniques that are being used in the aerospace and automobile industries?
What elements or methods will work in the AEC industries and what information needs/requirements can be
provided with more clarity than current methods?
This paper proposes that as the AEC industries move towards BIM and as the work force becomes dominated with
people who have grown up with Playstaion and Xbox 3D video games, we rethink methods of providing and
viewing complex building design intent that is not based on a 2D paper paradigm, but supplemented by. The cost of
printing and distributing paper documentation adds up to millions of dollars in lost revenue every yearnot to
mention the ecological impactwithin the AEC industries. Delays involved in physically distributing designs, the
difficulty of keeping distributed workers up to date, and the inability to track feedback on paper often result in costly
mistakes. Secondary and directly measurable costs for creating and delivering printed designs also remain.
(Autodesk, 2008)
The AEC have established digital file formats and systems to speed up a paper-based workflow that are independent
of the original authoring application for representing design data. Such as Design Web Format (DWF) developed in
the mid nineties by Autodesk for the efficient distribution and communication of rich design data, DWF was created
as a 2D only file format and has evolved to incorporate 3D data. The software developed to view the DWF files tries
to offer a Swiss army knife approach to accessing the model, providing every possible tool set to view, review and
mark-up, or print design information, resulting in confusing interface and complex learning curves. This technology
is based on a 2D paper based paradigm, resulting in problems of comprehension of printed 2D drawings.
The author wrote in a previous paper that Currently most CAD programs work within an object based mode,
using legacy methods and concepts from the drawing board. An architect or designer constructs a building or object
in digital space and then zooms in and out, rotating and panning around it. This type of navigation loses any sense of
scale, context and relationships between items. Gravity and sense of ground are also eliminated or suspended. These
representation modes impede comprehension of the building or object. Computer gaming software uses an
environment mode rather than an object mode to display architectural space, giving a viewer centric point or first
person perspective view. The shift from object based modelling to environment based modelling changes the way
the information can be viewed and understood, providing a scaled space which is navigated by walking or flying and
hyper-linking in real time. The first person view gives a stronger understanding of scale and relationships of the
proposed building. The current technology enables these spaces to be dynamic, with items which can be moveable,
even picked up and changed, using real world physics.
Ongoing research at a number of universities and companies around the world into the use of computer game-engine
technology in the AEC industries points to a possible direction to head. Autodesk labs, a new technologies
innovation centre, have announced a new technology tilted Project Newport real-time 3D story building
technology for architectural visualization and presentation. With game-engine technology and breakthrough ease-of-
use, Project Newport enables architects to show their design in context, rapidly explore design options, and create
vivid and immersive 3D presentations. Newport brings architectural designs to life by expressing design intent at
every stage of a project. A number of other industries are using and researching the use of computer game-engine
technology for information visualisations.
Research to date has started to show that an architectural hyper-model could be used to better understand
construction documentation to improve productivity. It has become clear the possibilities of this technology include:
A scalable view: the content can be viewed on any sized screen with little or no modification (projected
images, desktop and hand-held screens). Current 2D paper-based documentation is formatted for a single
size sheet that requires completely reformatting to be viewed on a hand-held computer or other platform.
Easy navigation control: based on standard computer gaming navigation conventions, in 2008, figures from
The Entertainment Software association of America showed that 65% of American households play
152
9
th

computer or video games, with the average game player age being 35. Current CAD software can be
confusing to navigate around due to its object based navigation focus.
Specification integration and material identification: with the use of realistic and coded textures that can be
directly linked to procurement and specification information. Creating hyper links between drawings and
specifications enable easy access between the model and non-model information or meta-data.
Fully Interrogative: unlike pre-rendered animated walk-throughs, the direction, the speed, and the sequence
of movement are not forcefully prescribed or scripted. This effectively eliminates the passive nature of the
animated walk-through and allows to the participants complete and thorough interrogation of the rendered
spaces. (Hoon et al., 2003)
Real-time physics: the use of a physics engine enables to the user real world physical properties within the
model providing to the viewer object collision with walls and floors, and the ability to have user controlled
moveable objects within the 3D environment.
Construction sequence visualization: Properly implemented 3D and 4D CAD systems can improve
communication of the construction process and therefore close the proverbial gap between the idea and the
implementation.(Saha and Hardie, 2004)
Integrated details: hyper-linking from model to details in real-time connected to a project database
providing fast and correct connections to details and further information.
Multiples: unlike physical drawings and models, a hyper-model can be viewed in multiple locations at once
(only requiring additional hardware).
Collaboration: multiple simultaneous users allow real time virtual meetings within the hyper-model;
facilitating designers, contractors and owners in different physical locations to meet in 3D digital space and
discuss issues within the context of the model.
Digital Document Management: the hyper-model would be stored online ensuring the latest model is
accessed by all and eliminating the potential for multiple versions, leading to miscommunication.
Due to the early stage of research of this project, with currently in true limitations, an architectural hyper-model on a
construction site have not be fully explored to date. It is expected that foreseen limitations can be managed through
design and formatting of interface per device or screen size requirements. More work is necessary to fully
understand these issues.
5. CONCLUSION
Advances in technology and the move towards building information modelling in the AEC industries are pointing to
a time to re-evaluate construction documentation methods. The processes used in the aerospace and automobile
industries demonstrate possible directions to begin the review. The maturing computer gaming industry is providing
powerful real-time 3D digital software that has potential to supplement current 2D paper-based construction
documentation. Providing an architectural hyper-model as part of the construction to improve understanding of the
design intent will improve construction efficiency. Future research will focus on developing methods and measures
to observe potential limitations and productivity gains of implementation of an architectural hyper-model into a BIM
workflow.
6. REFERENCES
Autodesk (2008). DWF: The best file format for published design information http://www.autodesk.com/dwf-
whitepapers.
Dale, E. (1969). Audiovisual methods in teaching, New York,, Dryden Press.
Eastman, C., Teicholz, P., Sacks, R. & Liston, K. (2008). BIM handbook: a guide to building information modeling
for owners, managers, designers, engineers and contractors, Wiley.
ESA (2008). Essential facts about the computer and video game industry. The Entertainment Software Association.
Gallaher, M. P., Oconnor, A. C., Dettbarn, J. L. & Gilday, L. T. (2004). Cost analysis of inadequate interoperability
in the u.s. capital facilities industry
153
9
th

Gallo, G., Lucas, G., Mclennan, A. & Parminter, T. (2002). Project documentation quality and its impact on
efficiency in the building and construction industry. Institution of Engineers Australia.
Gao, Z., Walters, R. C., Asce, M., Jaselskis, E. J. & Asce, A. M. (2006). Approaches to improving the quality of
construction drawings from owners perspective. Journal of Construction Engineering and Management.
Hoon, M., Jabi, W. & Goldman, G. (2003). Immersion, interaction, and collaboration in architectural design using
gaming engines. Proceedings of the 8th CAADRIA Conference. Indianapolis, IN, USA.
Issa, R. & Suermann, P. C. (2007). Evaluating the impact of building information modeling (BIM) on construction.
7 International Conference on Construction Applications of Virtual Reality:.
Kalay, Y. E. (2004). Architecture's new media : principles, theories, and methods of computer-aided design,
Cambridge, Mass. ; London :, MIT.
Pelosi, A. (2007). Architectural hyper-model: changing architectural construction documentation. Association of
Architecture Schools of Australasia. Sydney.
Pollalis, S. N. (2006). Understanding changes in architectural practice, documentation processes, professional
relationships, and risk management. gsd.harvard.edu.
Right-Hemisphere (2005). product graphics management for interactive 3d training.
Saha, S. & Hardie, M. (2004). The use of 3d and 4d cad systems to reduce error rate and reworking in construction.
Clients Driving Innovation Conference.
Thomas, M. (2004). A graphics pipeline for making 3D as cool as text. ACM SIGGRAPH Computer Graphics.
Tilley, P. A. (2005) .Design and documentation quality problemsa lean thinking opportunity. Proceedings of the
International SCRI Symposium.
154
9
th

CASE STUDIES ON THE GENERATION OF VIRTUAL ENVIRONMENTS
OF REAL WORLD FACILITIES
Michele Fumarola, Ph.D. Candidate,
Tel. +31 (0)15 27 89567
m.fumarola@tudelft.nl
http://www.tudelft.nl/mfumarola
Ronald Poelman, Ph.D. Candidate,
Tel. + 31 (0)15 27 88542
r.poelman@tudelft.nl
http://www.tudelft.nl/rpoelman
ABSTRACT: There is an increasing need to generate detailed real-time virtual environments that closely mimic real
world facilities. Drivers for this need are wider application of virtual training environments, new virtual pre-testing
of design, and joint virtual development of information systems. Approaches for generating virtual environments
vary from manual to automatic. In manual approaches we may have to go to the location, take pictures to develop
the 3D model and thereby construct the virtual facility. This approach is time consuming, inaccurate and coarse. In
automatic approaches we could use technology to capture the facility, e.g. laser-scanning, photogrammetric
methods and radar. Based on the data acquired a virtual environment can be generated automatically.
Unfortunately, automatically generated data sets are less than optimal for practical use within real-time virtual
environments because of the huge unstructured amount of data. Common approaches are therefore most likely to
have a balance between human and computer effort. Computers are typically suitable to do bulk work, where
humans are well suited to pick-up unnatural phenomena. Based on different case studies, we discuss the distribution
of manual and automatic methods for the generation of 3D virtual environments. Different facets of the pipeline
from initial data gathering up to a final deliverable are presented. The case studies vary from fully hand made up to
semi automatic reconstruction of the environments. The paper concludes with recommendations regarding the
reconstruction methods.
KEYWORDS: Virtual environment, virtual reconstruction, visualization, serious gaming
1. INTRODUCTION
1.1 A need for 3D virtual environments based on real world facilities
There is an increase in use of 3D virtual environments in architecture, engineering and construction (AEC)
industries. Applications like virtual training environments, virtual pre-testing of design, and joint virtual
development of information systems require a valid representation of the real environment. Often, the real
environments are industrial facilities such as oil rigs, containers terminals, or manufacturing plants.
There is no single reason why 3D virtual environments are increasingly popular, but drivers stem from multiple
backgrounds. From a theoretical perspective advantages of 3D virtual environments are found in improving
communication (Arayici and Aouad 2004), increasing insight (Shiratuddin and Thabet 2007; Woksepp and Olofsson
2006), supporting collaboration (Bouchlaghem et al. 2004), and supporting decision-making (Kam and Fischer
2004). The divergence in applications requires different levels of fidelities of the 3D virtual environment. This can
be illustrated by the different levels of fidelity required in the design process of a manufacturing plant. Designing the
plant is mostly done in a 3D environment with high precision. The design drawings are complex, show different
layers (e.g. mechanical, electrical, and plumbing) and therefore become hard to understand. On the contrary, for the
155
9
th

presentation of the final result to stakeholders, a 3D visualization with a reduced level of complexity is preferred.
The different types of visualizations achieve different types of fidelity for each specific goal.
1.2 Realism in 3D virtual environments
According to Websters dictionary, fidelity means the accuracy in details. While fidelity is the general term for the
way in which a model is a valid representation of a reference system, 3D modellers tend to use the term realism, as
their reference system is the real world, i.e. the real industrial facility. Ferwerda (2003) distinguishes three varieties
of realism in computer graphics: physical realism, photo-realism, and functional realism. For each type of realism,
there is a criterion which needs to be met in order to achieve that type of realism.
Physical realism is achieved when computer graphics provide the same visual stimulation as reality. This type of
realism means that the image has to be an accurate point-by-point representation of the spectral irradiance values at
a particular viewpoint in the scene. It requires an accurate description of the scene, simulation of the spectral and
intensive properties of light energy and reproduction of those energies by the display device. Technically this type of
realism is the hardest one to achieve. While this aspect is often ignored for more interpretational and overview
models that are geared towards human observers, it might become essential in 3D virtual environments for future
use. In this paper physical realism is ignored.
Photo-realism in a virtual scene provides the same visual response as the real scene. It aims at displaying an image
indistinguishable from a photograph of the real scene. Although achieving photo-realism has primarily been a task
for off-line rendering algorithms, modern interactive 3D visualization software libraries tend to surpass the vague
threshold towards photo-realism.
Functional realism is about providing the same visual information as found in reality. The main concern of this type
of realism is about transferring information about the real objects such as their shapes, sizes, motions and materials.
As this type of realism aims at providing the necessary information to perform visual tasks, functional realism can
be found in photo-realistic images as well as in simplistic sketches. Measuring whether functional realism is
achieved is therefore a challenging task as it requires an interpretation from human observers. The functional realism
aimed for in this paper is what engineers would call accuracy.
Although the three types of realism are compared against reality and real environments, a realistic 3D
environment is not necessarily one found in the real world. A highly populated city could be displayed photo-
realistically although the buildings and layout have been procedurally generated (e.g. Whelan, Kelly, & McCabe
(2008)). In virtual environments for AEC the common practice is to model existing facilities or facilities under
design or construction. When under design or construction, a valid reconstruction of the future facility is required
before it can be displayed. Several methods and techniques are available to perform these reconstructions. The key
aspect in which they differ is the level of automation. Some methods require a completely manual reconstruction.
Other methods however facilitate that data can be gathered automatically thus leaving everything up to algorithms
which can be run on computers.
1.3 Human and computer effort in the reconstruction of virtual environments
Building a 3D virtual environment based on real world data is a tedious task. In Poelman & Fumarola (2009), we
discussed which technologies can be used and which skills are needed. The question remains to what extent
automation is feasible and appropriate in building virtual environments of real world facilities. This question has
come forward in multiple studies resulting in different approaches. These approaches are mainly situated in one of
the following four groups: geometry based, image based, point cloud based and hybrid approaches (Baltsavias 1998;
Debevec et al. 1996; Hung 2007; Tao 2006; Zlatanova 2008).
In the gaming and animation industry, manual 3D modelling is the dominant approach. This approach, known as
geometry based approach, is one of the most labour intensive ones. It therefore scales badly to large or detailed
environments in which each new object has to be modelled separately. Some level of automation is possible in this
approach, mainly making variations on objects and environments. The skills of the designer are essential for the
quality of the result.
Image based modelling approaches aim at retrieving 3D models from a single or multiple 2D images of a physical
object (Debevec et al. 1998; Hartley and Mundy 1993; Remondino and El-Hakim 2006). The typical workflow starts
156
9
th

with the retrieval of (calibrated) pictures. Then geometry is extracted from these pictures. Once the geometry has
been extracted, textures based on the initial pictures are applied. This approach has shown promising results but is
still under research (some recent advances are described in Azevedo et al. (2009), Gruen (2008), and Sinha et al.
(2008)).
The point cloud based approach uses capturing devices capable of recording 3D points in real environments.
Capturing devices are known as active sensors (in contrast to passive sensors, e.g. camera, used in image based
modelling) and laser scanners are a well known example. Active sensors are split in two categories: ground laser
scanners and airborne light detection and ranging (LiDAR) sensors (Hu et al. 2003; Wehr and Lohr 1999). Due to
the advanced equipment and setup, this approach is relatively expensive and difficult to use. The result that can be
achieved are, highly detailed 3D models (e.g. Dellepiane et al. (2008)).
Hybrid approaches use a combination of the aforementioned approaches to surpass some of the disadvantages found
in using a single approach. In most cases, these disadvantages are costs and capabilities of a particular approach.
Although image based and point cloud based approaches are near fully automated, they still require a lot of human
effort. In image based approaches, taking images need to adhere to specific requirements, and postproduction is a
necessity. Point cloud based approaches provide huge datasets which cannot be used directly in virtual
environments. The level of automation is case-specific and approaches are tailored towards requirements coming
forward from that case.
2. CASE STUDIES
2.1 Presenting the case studies
We present four case studies that modelled a real world facility in a 3D virtual environment. The mix of manual and
automated approaches varies between all four case studies: we start with a case study where no automatic generation
took place and end with one that was almost generated automatically. Impressions of the case studies are shown in
Figure 1.
For each case study, we start with a general introduction and the requirements set in terms of what type and level of
reality we aimed for. We continue by discussing the trade offs we had to make in terms of effort, automation and use
of existing data. Finally we discuss the choices made and present how the implementation took place and what the
results were.
2.2 Supervisor training
2.2.1 Introduction
The conventional way of training supervisors for the petrochemical industry combines hands-on training in training
facilities and the use of videos, slides and questionnaires. Experiential learning, as brought forward in serious
games, is assumed to improve the knowledge retention over conventional learning methods. A pilot has been
conducted at a large oil company to explore the benefits of 3D virtual training / serious gaming for training
supervisors in the petrochemical industry. The objective is not to replace current training solutions but to position
serious gaming as an additional teaching method. To do so, we developed an immersive 3D serious game that was
based on an actual training location.
2.2.2 Requirements
The 3D environment is based on an existing location in use for real life training. Therefore a key requirement of the
virtual counterpart was achieving a high degree of photorealism with the existing location. Artificial defects were
going to be integrated and had to blend in the environment.
The following requirements were proposed for the 3D environment:
The 3D environment has to look photo realistic; spatial correctness is less important than visual
appearance.
Details that are of importance to the scenario have to be homogeneous to the rest of the environment, so
they will not be easily recognizable.
157
9
th

When possible, use the photographic information gathered during acquisition for the texturing process.
Weather conditions should be changeable.

FIG. 1 Impressions of the case studies: Supervisor in the upper-left corner, the virtual terminal in the upper-right
corner, the chemical installation in the lower-left corner, and the off-shore platform in the lower-right corner.
2.2.3 Trade offs
Due to the high amount of detail to be generated, we choose a site with a compact size. The site was manually
modelled based on photographs without rectification or orientation. Therefore the models were approximations of
the actual objects with regards to dimensions. Nevertheless they appeared realistic because of the textures that were
based on actual photographs.
Most of the detailing was focused on the objects and locations that were of importance to the scenario. Specific
details take a lot of time to create whereas stock content can be used almost instantly. Therefore the environment
outside the compound (drilling site) was neglected and the buildings that were of less importance were modelled
generically.
Manual 3D modelling was used in favour of automatic acquiring techniques because the pipeline for creating the
content for this particular industry and client was just getting started. With no awareness of the manual procedures,
it is very hard to correct mistakes in automatically generated content.
158
9
th

2.2.4 Implementation
The environment was implemented in Unreal Engine 3
1
using the development environment that comes with the
engine. The layout of the site was created with the included editor. The details however, which are harder to model,
were created with specialized 3D modelling tools. As the facility consisted mainly of piping, a component based
approach was created which allowed us to reuse existing models to assemble the complete environment.
Due to the large amount of photographic material used in the environment, approaches had to be found to reduce
memory issues. Procedural textures were therefore used to reapply them on multiple models.
2.2.5 Results
The immersive 3D virtual training environment / serious game demonstrated the power of serious gaming for
training supervisors. Although a thorough evaluation has yet to be performed, the first informal evaluations show
some promising results.
Approximately 40 people provided feedback with regards to the environment and were overall positive. People
familiar with the site directly recognised it up to the details. Most people even assumed the environment was
spatially accurate although it was only approximately correct.
Because manual modelling was the selected method a lot of man hours were used to create the environment. It took
two people 3 full months to just model the environment. The amount of time needed to provide this much detail with
manual methods raises the question if there was no speedier and more cost-effective way to achieve these kinds of
results.
2.3 Automated container terminals
2.3.1 Introduction
Novel modes of operations have to be developed to handle the increasing numbers of containers at modern
terminals. Automated handling equipment provides a solution and is becoming the default choice for the design of
new container terminals across the globe. However little experience and knowledge is available on how to design
this new type of container terminals. Virtual environments provide the possibility to look into the future and
visualize container terminals that do not yet exist. This enables designers to achieve insight, communicate their
designs and experiment with different layouts. We designed a support tool for the design process.
2.3.2 Requirements
The container terminals to be visualized in the virtual environment do not yet exist, ruling out the use of acquiring
technology such as laser scanners. The major source of data consists of the CAD drawings made by designers and
photographs of equipment that will be used in the future terminal.
The virtual environment has to be automatically generated from the CAD drawings. As the virtual environment will
be used throughout the design process of the new terminal, the drawings will be changed and updated regularly,
leading to updates in the 3D virtual environment. This prohibits custom modelling work for the 3D environment as it
will be overwritten with each update.
The following requirements were identified:
Automatic generation of the virtual environment from the layout CAD drawings.
Spatial precision in respect to the CAD drawings.
Precise 3D models of the equipment.
Visual appearance was important for possible presentations.

1
Website at http://www.unrealtechnology.com, accessed on July 14
th
, 2009
159
9
th

2.3.3 Trade offs
To automatically generate the virtual environment, a component based approach was used for the translation from
CAD to VR. The downside of this approach was having the same models duplicated multiple times, resulting in a
decrease of realism as real equipment often differs in details like rust and damage.
Customization based on key features of locations (e.g. environment outside of the terminal) is also not considered in
this approach. As the design process of such terminals takes up to a year (or more), the possibility of developing
custom partial models can be considered. This would give the designers an idea of the real world environment in
which they are operating.
The implementation consists of an Autodesk AutoCAD plug-in and a stand alone application developed using
OGRE for the 3D visualization. The AutoCAD plug-in handles the translation from CAD to VR while the stand
alone application visualizes the 3D environment.
Using a predefined convention based on blocks in AutoCAD, a translation to an intermediate XML format has been
made. This approach, based on the library based approach as described in Whyte et al. (2000), was found to be
appropriate as a large amount of objects are reused throughout different terminals. As the CAD drawings use
millimetres as the main unit, the resulting virtual environment could benefit from this precision.
The stand alone application reads the intermediate XML file to construct the virtual environment. The 3D models in
the virtual environment have been modelled based on CAD blueprints of the actual equipment which resulted in
precise models. Photorealism was further aimed for by using photographs of existing equipment to texture the
models. Photographs were also used for the different types of materials found on a virtual terminal: concrete,
tarmac, metal, etc. The environment was further enriched using aerial pictures.
2.3.5 Results
The resulting virtual environment serves as a support tool in the design process of automated container terminals.
The virtual environment can easily be changed by modifying the CAD drawing. It serves as a knowledge sharing
platform for this novel type of terminals.
Although photorealism is less important than functional realism, the level of photorealism present in the virtual
environment ensures everybody has a clear understanding of the new container terminal. Moreover, using the virtual
environment as a presentation tool helps communicating future developments outside of the company (e.g. fairs and
professional congresses).
2.4 Chemical installation
2.4.1 Introduction
Europe is densely industrialized: the most appropriate locations for industry are taken which leaves little choice for
new industrial sites. Therefore brownfield engineering is more common than greenfield. This implies that knowing
the state of installations is of crucial importance when revamping a site or when maintenance has to take place.
A lot of installations have been designed and built decades ago: if engineering data is available, it is in most cases
too old, incomplete or wrong. Nowadays computer programs are being used to handle 3D engineering data to
support the replacement and design processes. For a proper job, good reference data is essential.
We had a strong need for accurate and complete 3D reference data for a large revamping project. Due to the
complexity of the task at hand the model would have to be ready for interaction. A chemical pipeline of 8 inches
needed to be routed from a storage vessel to a production plant for which it had to cross two pipe racks, a road and
another production facility.
2.4.2 Requirements
The following requirements were identified:
A complete 3D representation of the areas the pipeline has to cross with a tolerance of 2 mm for connecting
flanges and 5 mm for the rest of the model.
160
9
th

All crossing structural steel, piping and other objects needed to be present for clash checking.
The individual objects needed to be aligned with the engineering grid.
2.4.3 Trade offs
Laser scanning was chosen over traditional surveying because of the accuracy and completeness that laser scanning
can provide. This means there was some level of automation in the 3D acquisition process. Because the engineering
software that was used in the project could not interact with point clouds, the entire point cloud had to be translated
to solid geometry. The modelling process itself could not be automated at that moment in time.
There were problems in previous projects with cloud-fitted models. In reality most things are a little tilted, offset or
damaged which is reflected in the cloud fitted models. However, most engineering packages handle their models as
being perfect. By bringing in slightly imperfect models into the engineering software things go wrong: e.g. incorrect
centrelines, slopes, and elbows. Because of previous experiences, the decision was made to create the models within
the engineering software with the scans as a reference instead of being depended on cloud fitting geometry
algorithms.
The Leica HDS 3000
2
was used to acquire 79 scans which were geo-referenced for accuracy and control of the
process. Because the scans were geo-referenced, they could be semi-automatically registered. Although it takes
more time on location, it is for many survey companies the preferred way of working.
Furthermore the scans were modelled by loading them in a piping specific engineering application. They were traced
to acquire a solid model; an approach that is somewhat less accurate then cloud fitting, but the engineering
environment was not able to cope with intelligent objects combined with deviating geometric forms.
The engineering of the model was done by experts: the decisions involving the differences between laser scans and
CAD models could therefore be settled on the spot.
2.4.5 Results
The model was used to interactively create the routing of the piping. By using laser scans as a base for the design
process the time for the entire project has been reduced. Comparing the throughput time with the conventional
planning schemes showed time reduction of approximately 25%.
Less automated geometrical data acquisition would have been difficult because of the divergence and complexity
found in the environment for the pipe routing process. The manual modelling process used in this case study,
resulted in a tedious task to create the 3D models.
2.5 Off-shore platform
2.5.1 Introduction
For this case two methods were used to construct the 3D environment of an off-shore platform. The same part of the
environment was modelled twice: once semi-automatically based on laser scanning and once using fully automatic
modelling also based on the same laser scans. This was done to compare the two results in order to be able to pick
the best result. Part of the results of the automatic modelling process are described by Rabbani and van den Heuvel
(2004). Because this example (laser scanning and automatic modelling) provides insight into an almost fully
automated creation process we discuss the results from a different perspective.
For engineering purposes an accurate 3D model of an off-shore platform had to be generated. Some of the piping
needed to be replaced, preferably a one-on-one replacement. Because labour on an off-shore platform can be more
dangerous and expensive than on-shore, prefabrication of parts is desirable. Moreover the 3D model had to be used
to find the best decommissioning and reattachment of new parts as the piping that needed replacement is
interweaved with the rest of the piping.

2
Website at http://hds.leica-geosystems.com/en/5574.htm, last accessed on July 14
th
, 2009
161
9
th

2.5.2 Requirements
The model had to be generated for engineering purposes; therefore accuracy is of major importance. The piping is
one to three inch in diameter, which is this case means approximately five millimetre accuracy. All objects that
could provide a clash with the decommissioning or reattachment needed to be represented in the 3D model.
An additional requirement was set by the tight schedule the project was on. Due to this, development speed was
preferred over costs, which provided an additional challenge.
2.5.3 Trade offs
Because of the high accuracy standards used in engineering and the tight schedule, laser scanning was the preferred
choice of technology. Conventional surveying could however provide the same models although it would have been
too slow.
Although laser scans can be registered automatically, it was decided to do a manual registration process to have
control over the registration process, because of the uncertainties of automatic methods. For the modelling process
the decision was made to semi-automatically create the 3D model. A fully automatic modelling process followed
afterwards to compare the results.
To get comparable results between semi-automatic and fully automatic modelling, a single room of the offshore
facility was modelled. The rooms approximate length was 12 meters wide, 6 meters long and 4 meters high. There
were 32 scans created to get 80-90% coverage of the room whereas all critical components had 100% coverage.
Control of the scanner and the total station required 2 people.
The semi automatic modelling process was done using region growing algorithms (Rabbani and van den Heuvel,
2004) using specialised point cloud reconstruction software. Solid geometry was generated and exported so it could
be used within a engineering CAD viewer. The viewer was used to interact with the 3D environment so decisions
for spool sizes, decommissioning and fitting could be made between the experts.
2.5.5 Results
The time needed to get the scans done was approximately 4 hours using a Leica HDS 3000. The registration process
took approximately double that time. The registration parameters were all below 5 millimetres. The experts on
location estimated that this would be at last a week to do manually.
The semi automatic modelling process took 15 days to create 2602 objects (planes, cylinders), counting the complex
geometry in planes. In comparison, 2338 objects were detected fully automatically although there were quite some
subdivided objects e.g. round columns could be detected multiple times because of missing information in the point
cloud, slicing up the dataset. The points that were part of the segmentation process were approximately 80% of the
total amount of points whereas the points on the generated objects, 53%. This resulted in 946 planar patches and
1392 cylinders (Rabbani and van den Heuvel, 2004). An estimated 40% of the semi-automated modelled objects
were directly usable.
3. REFLECTIONS AND CONCLUSIONS
The case studies presented show an increase of the level of automation possible in the acquiring and modelling
process for the reconstruction of virtual environments. We started with the discussion of Supervisor which was
modelled manually. We continued with the virtual container terminal which consisted of building a component
library and using this library to automatically construct virtual versions of multiple container terminals around the
world. Thirdly, we discussed a case study wherein a chemical installation was reconstructed using laser scanners but
due to scene complexity, manual models were made based on the point clouds gathered from the laser scanners.
Finally, we presented the case of the off shore platform in which a high level of automation was achieved.
The choices for the different approaches were based on the properties of the real environment and on the goals of the
virtual environment. The real environment can vary from being small to large and from being simple to complex.
The goals of virtual environment can result in the need for photorealism, the need for functional realism, or both.
162
9
th

Based on these properties and goals, we will generalize approaches from the case studies conducted. A summary of
these approaches with the properties and goals can be found in Table 1.
Whenever a virtual reconstruction of an environment has to be made, size and complexity of the environment play
an important role. Although automatic techniques give the possibility to acquire spatial information rather quickly,
modelling can be troublesome due to the large amount of data gathered. This data needs to be modelled according to
the possibilities of the visualization engine wherein it is going to be used. When using automatic techniques such as
laser scanning, the resulting point clouds tend to be huge, making it impossible to use it in real-time environments.
To reduce the amount of information, specific algorithms can be used. Nevertheless, these algorithms do not give
optimal result for complex scenes as for the chemical installation case study where a huge amount of pipes and small
details were present. In these cases, manual intervention is still required, reducing the level of automation possible in
the modelling phase. In less complex scenes, algorithms for mesh simplification are better suited for the task.
As using methods for automatic acquiring and modelling is still labour intensive and requires specific skills, the
choice for manual methods is still often considered. Automatic acquiring methods result in a high level of accuracy
(up to a millimetre), thus functional realism, which is not always needed. In training environments, such as
Supervisor, photorealism is preferred whereas accuracy is not even required. In these cases, manual modelling gives
the possibility to fully concentrate on details using a team of artists instead of skilled engineers.

CASE SIZE COMPLEXITY FUNCTIONAL
REALISM
PHOTOREALISM ACQUIRING MODELLING
Supervisor
training
Small Low Low High Manual Manual
Automated
container
terminals
Large Low High High Manual Automatic
Chemical
installation
Small High High Low Automatic Manual
Off-shore
platform
Small Low High Low Automatic Automatic
Table 1 Summary of conclusions from the case studies from the perspective of the properties of the real environment
and the goals of the virtual environments.
Choosing between manual and automatic methods for the acquisition and modelling phase for the reconstruction of
a virtual environment, is not straightforward. Trade offs have to be made based on the environment that is being
reconstructed and the requirements set for the virtual environment. Although automatic methods are gaining
popularity thanks to their accuracy and speed, human intervention is still often needed. Complete automatic methods
can only be used in ideal circumstances, which are less common than one would wish for.
4. ACKNOWLEDGEMENTS
The authors would like to thank APM Terminal Management BV for their input and support on the second case
study.
5. REFERENCES
Arayici, Y., and Aouad, G. (2004). "DIVERCITY: distributed virtual workspace for enhancing communication and
collaboration within the construction industry." eWork and eBusiness in Architecture, Engineering and
Construction, A. Dikbas and R. Scherer, eds., Taylor & Francis Group, London.
Azevedo, T. C. S., Tavares, J. M. R. S., and Vaz, M. A. P. (2009). "3D Object Reconstruction from Uncalibrated
Images Using an Off-the-Shelf Camera." Advances in Computational Vision and Medical Image
Processing, J. M. R. S. Tavares and R. M. N. Jorge, eds., Springer Netherlands.
163
9
th

Baltsavias, E. (1998). "A comparison between photogrammetry and laser scanning." Journal of Photogrammetry
and Remote Sensing, 54(2-3), 83-94.
Bouchlaghem, D., Shang, H., Whyte, J., and Ganah, A. (2004). "Visualisation in architecture, engineering and
construction (AEC) " Automation in Construction, 14(3), 287-295
Debevec, P., Taylor, C. J., and Malik, J. (1996). "Modeling and rendering architecture from photographs: a hybrid
geometry- and image-based approach." 23rd Annual Conference on Computer graphics and interactive
techniques.
Debevec, P., Taylor, C. J., Malik, J., Levin, G., Borshukov, G., and Yu, Y. (1998). "Image-based modeling and
rendering of architecture with interactivephotogrammetry and view-dependent texture mapping." IEEE
International Symposium on Circuits and Systems, Monterey, CA, USA.
Dellepiane, M., Callieri, M., Ponchio, F., and Scopigno, R. (2008). "Mapping Highly Detailed Colour Information
on Extremely Dense 3D Models: The Case of David's Restoration." Computer Graphics Forum, 27(8),
2178-2187.
Ferwerda, J. (2003). "Three varieties of realism in computer graphics." Human Vision and Electronic Imaging VIII,
T. N. Pappas and B. E. Rogowitz, eds., Santa Clara, California, USA.
Gruen, A. (2008). "Reality-based generation of virtual environments for digital earth." International Journal of
Digital Earth, 1(1), 88-106.
Hartley, R. I., and Mundy, J. L. (1993). "Relationship between photogrammmetry and computer vision." Integrating
photogrammetric techniques with scene analysis and machine vision, Orlando, Fl, USA.
Hu, J., You, S., and Neumann, U. (2003). "Approaches to large-scale urban modeling." IEEE Computer Graphics
and Applications, 23(6), 62-69.
Hung, Y.-P. (2007). "An Image-Based Approach to Interactive 3D Virtual Exhibition." Lecture Notes in Computer
Science, 4872, 1.
Kam, C., and Fischer, M. (2004). "Capitalizing on early project decision-making opportunities to improve facility
design, construction, and life-cycle performance: POP, PM4D, and decision dashboard approaches."
Automation in Construction, 13(1), 53-65.
Poelman, R., and Fumarola, M. (2009). "Technology and skills for 3D virtual reality serious gaming: "Look before
you leap"." 40th International Conference of the International Simulation and Gaming Association,
Singapore.
Rabbani, T., and van den Heuvel, F. (2004). "Methods for Fitting CSG Models to Point Clouds and their
Comparison." Computer Graphics and Imaging.
Remondino, F., and El-Hakim, S. (2006). "Image-based 3D modelling: A review." The Photogrammetric Record,
21(115), 268-291.
Shiratuddin, M. F., and Thabet, W. (2007). "Information Rich Virtual Design Review Environment." 4th W78
Conference, Maribor 2007 & 5th ITCEDU Workshop & 14th EG-ICE Workshop.
Sinha, S. N., Steedly, D., Szeliski, R., Agrawala, M., and Pollefeys, M. (2008). "Interactive 3D architectural
modeling from unordered photo collections." ACM Transactions on Graphics, 27(5), 1-10.
Tao, C. V. (2006). "3D Data Acquisition and Object Reconstruction." Large-scale 3D data integration, S. Zlatanova,
S. Zlatanova, and D. Prosperi, eds., CRC Press.
Wehr, A., and Lohr, U. (1999). "Airborne laser scanning - an introduction and overview " ISPRS Journal of
Photogrammetry and Remote Sensing, 54(2-3), 68-82.
Whelan, G., Kelly, G., and McCabe, H. (2008). "Roll your own city." 3rd international conference on Digital
Interactive Media in Entertainment and Arts, Athens, Greece.
Whyte, J., Bouchlaghem, N., Thorpe, A., and McCaffer, R. (2000). "From CAD to virtual reality: modelling
approaches, data exchange and interactive 3D building design tools " Automation in Construction, 10(1),
43-55.
Woksepp, S., and Olofsson, T. (2006). "Using virtual reality in a large-scale industry project." The Journal of
Information Technology in Construction, 11, 627-640.
Zlatanova, S. (2008). "Data collection and 3D reconstruction." Advances in 3D Geoinformation Systems, P.
Oosterom, S. Zlatanova, F. Penninga, and E. M. Fendel, eds., Springer Berlin Heidelberg.

164
9
th

EVALUATION OF 3D CITY MODELS USING AUTOMATIC PLACED
URBAN AGENTS
Gideon D. P. A. Aschwanden,
Chair for Information Architecture, Switzerland;
aschwanden@arch.ethz.ch
Simon Haegler,
Computer Vision Laboratory, Switzerland;
shaegler@vision.ee.ethz.ch
Jan Halatsch,
stefan.arisona@arch.ethz.ch
Rafal Jeker,
rjecker@ee.ethz.ch
Gerhard Schmitt, Prof.,
schmitt@arch.ethz.ch
Luc van Gool, Prof.,
vangool@vision.ee.ethz.ch
ABSTRACT: We present a method for populating procedurally generated 3D city models with crowds of artificial
agents. It is targeted towards the analysis, prediction and visualization of occupant behaviour in urban planning.
We simulate and quantify correlations on the following aspects: functions of buildings, number of people and
fluctuation in density. Potential practical applications are for example a) to determine bottlenecks in public transit,
b) to identify possible problems for evacuation scenarios, c) to evaluate the demand for and the accessibility of
amenities as well as d) the stress of pedestrians to evaluate quality of life indicator for a given urban region . The
occupants location data represented by the agents - and relevant semantic metadata are encoded inside a
grammar-based city modelling system. This information is used for the context-dependent automatic placement of
occupant locators during the procedural generation process of the urban 3D model. Most of the underlying
parameters are interconnected with each other. For example, the number of resulting agents corresponds to the size,
function and location of one specific building. Once a 3D city model has been generated, occupants are represented
by agents using a) a commercial fuzzy logic system and b) pre-animated 3D avatars. The agents find their way
through the city by moving towards points of interest to which they are attracted. Each individual agent draws
specific paths while interacting with the urban environments and other agents. Every path describes a set of
parameters, for example speed, space available and level of exhaustion. The ensuing visual diagrammatic
representation shows the resulting agent paths in correlation with the virtual environment. This offers the
opportunity to investigate parts of a city and optimise corresponding aspects with minimal interventions on the
urban level. We show the application of this method to evaluate planning interventions in the urban fabric and
monitor the correlating effects.
KEYWORDS: Artificial intelligence, agent-based, crowd simulation, Space Syntax, urban planning, design
evaluation, occupant movement.
165
9
th

1. INTRODUCTION
The number of planners who come to the conclusions that urban plan should be more aligned to the needs of the
pedestrian occupant is increasing. It seems quite clear that the more an urban design works out as a performing
environment for pedestrians including infrastructures such as public transport or pedestrian miles the less
consumption of energy and emission of CO
2
will be performed in private transportation. Therefore we like to
present a new approach to evaluate visually how urban designs perform in a pedestrian context. Our method
combines the robust techniques of commercial crowd simulation applications with the iterative nature of procedural
city modelling techniques in order to have a) fast as well as understandable visualizations and b) a effective design
tool at hand that enables the designer to perform the comparison of design alternatives very quickly. This method
offers as well an added value for the entertainment industry while delivering high quality imagery output through
standard production pipelines and a decreasing workload the procedurally generated urban layouts that are simulated
as a potentially realistic urban environment with associated virtual occupants. Traditionally, costs and time needed
to produce populated digital urban sets are enormous for movie, game and interactive VR projects.
1.1 Related Work
In the following we give a short outline of selected works related to the simulations of groups and crowds. For a
more comprehensive description we refer the reader to (Magnenat-Thalmann et al., 2004). Models for group
behaviour have been an active research field since the late 19th century, for example LeBon (1895). Todays
computer simulation models have a relatively young history. Most relevant approaches have been realized within the
last 20 years and are specialized as a specific solution for different fields.
More specifically is Reynolds (1987) flocking method, which uses particle systems and represents one of the most
common approaches for simulating group movement within the entertainment industries. Brooks (1991) provides a
comprehensive foundation on which many of the recent agent models and theories are based. He describes many
failing artificial intelligence approaches to set-up intelligent agents. Musse and Thalmann (2001) introduced a more
flexible model with hierarchical behavior. Physics and body effects had been described by Helbing et al. (2000) to
simulate escape panics effectively. In other fields like robotics (Molnar and Starke 2001), safety science (Still 2000)
and sociology (Jager et al. 2001) similar approaches have been created simulations involving groups of individual
intelligent units. Despite the actual size of the crowd simulation research basis interdisciplinary exchange between
groups of researchers is relatively limited. Hillier and Hanson (1984) introduced the idea that a city and spaces in
general can be divided into components to analyze them as a network of choices and be represented as maps and
graphs. Penn and Turner (2001) then described their method to use urban agents within their space syntax system.
Parallel systems evolved in computer graphics to populate urban environments in real-time (Penn und Turner, 2001;
Tecchia and Chrysanthou, 2000). In Aschwanden et al. (2008) we introduce our occupant simulation method, which
is now extended by the present work. The presented procedural modelling technique has been initially presented by
Parish and Mueller (2001) for the modelling of cities, by Mueller et al. (2006) specifically for the modelling of
buildings and has been practically applied to the context of urban planning by Ulmer et al. (2007) and Halatsch et al.
(2008a and 2008b).
2. URBAN MODEL
The authors interpret the term the city as a complex, distributed, interconnected and rapidly changing system that
can be currently understood as a fabric of space depending on a) People, b) Function, c) Space, and d) Physical
Environment
Each aspect listed above is typically under the influence of a number of variables and forces. As urban planners, our
aim is to try to control both the layout and the functioning of the urban system, while regarding the most influential
variables of a given urban situation through our empiric planning knowledge. In the case of a complex city space,
the occupants may either accept the designers work or not, and their subsequent use and acceptance of the designed
urban space defines a successful urban planning. In urban planning there exist many important correlations. For
example, on one side economic demand asks for more high-density buildings in city centers, which coexists with the
commuters demand to have access to green open spaces. These contradicting desires call for an equilibrium, which
must be recalculated constantly. Our goal is to satisfy both; the economic and personal needs of people. It requires
us a) to understand the given situation, b) to indentify the necessary semantic metadata, c) incorporate insights into
166
9
th

the procedural 3D model and the agents model, and d) to run the simulation. On a very abstract level, the output
presents vital clues about possible human acceptance of the given configuration of a city. The quantifiable judgment
of the agents, while rather simple and predefined, seems to be very accurate. We are not implying that one agent
alone can provide us with all of the information we need, but by combining hundreds or thousands of agents within
the one setting we create a kind of group intelligence.
3. URBAN DENSITY
3.1 What Kind of Density?
Urban densities are under constant discussion - the factors by which to classify density are not clear. In addition, the
output value does not have a clear unit. Some urban planners find that the number of occupants per square kilometer
is a useful figure. Another useful figure may be the number of cars recorded between urban centers, but these two
figures have already very little correlation. A major achievement is to have a correlation between the built
environment, number of people and traffic. Our contribution is the enhanced level of detail, in contrast to the
aforementioned examples of density; we use the existing plots in combination, with those being developed, and take
note of buildings already erected on these plots to predict the number of people likely to be occupying that area.
Other calculation methods, like Arnott and Small (1998), use distant functions from a virtual centre or a grid overlay
on the city. This roughness allows arbitrariness in both of these methods.
Our approach (see figure 1) is pragmatic, because we want to know how many people are coming out of the building
and are walking the streets. There are several aspects involved; three of them have a major influence: a) the floor
size of the building, b) use function and c) the location of the building. These factors enable us to predict, how many
people are coming out of a building at a certain moment in time. Verified statistical data in a reasonable level of
detail is key to comprehensive results. Until now we only have input data from Switzerland what is the limitation to
our approach.
FIG. 1: Human Locator (Input-Output based calculation method for the prediction of the number of people coming
out of a building).

3.2 Calculation4
3.2.1 Source of data
We used the Swiss census data by the Swiss government to define the floor space per person in a building living
area as well as the space requirements for offices and workspaces. The overall number of occupants is not decisive
enough, only in combination with the fluctuation over time were able calculate the people moving in and out of the
building. These factors are very close related to the characteristics of a place. We automated the calculation and
specified it according to every commune in Switzerland (Diener, 2006). This was necessary due to the high disparity
of the input data.
3.2.2 Example of a residential house
For residential areas we used the floor space of the apartments and divided it by number of residents to get the
average space available to every resident. With normal fluctuations arousing from people moving, the housing
167
9
th

market is not operating at full capacity. Therefore an economical approach (Wst und Partner,
http://www.wuestundpartner.com/) is also incorporated. The combination of net floor space, space available,
location and the fluctuation over time allowed us to predict the traffic produced by every building. The limits of
such automated calculation is reached in cases where the traffic generated is not a stable function, such as a stadium
emptying itself with 30 min 2 times a week or when the number of buildings is to small to balance out the statistical
imprecision.
3.3 Preliminary Findings
The available average floor space for residents is varies in Switzerland from 35.5m
2
(Zrich) to 55.7m
2
(Maisprach).
Both numbers are higher then the space provided for office workstations from a minimum 4.46m
2
for secretarial
workstations to 27.89m
2
for a vice-president office. This shows that the traffic generated by office buildings is much
higher then the residential buildings in general and is also more volatile over time. The 3
rd
places, such as
restaurants, cinemas as well as hospitals, produce an even higher number within their peak time but are difficult to
predefine.
4. AGENTS
The agents that we used consist of perceptual, behavioral and cognitive components in combination with a physical
body to test the practicality of our 3D city model. Every agent represents a measuring device by walking in the city
using its function channels to learn about the environment and the artificial brain to process it and give us
information about the path it took. The agents are able to see, hear its environment and read information that has
been written to the ground of the 3D environment. These abilities are processed inside the artificial intelligence
logic by each agents instance that is present in the 3D environment for evaluation. This enables the agents to
interact with their surrounding, their built environment as well as with other agents. A unique set of personality
characteristics and abilities for each agent allows this collection of individual bodies to be a realistic representative
of a standard population. The agents entity of body and brain is not only a shape; it merely is a semantic organism
enabling the agent to interact with distinct features and abilities.
4.1 Sensory Channels for Perception
We are using multiple sensors as receptors for the agent to understand its environment. Signals from a single object
are affiliated by a sensory system. Like humans the agents have a limited set of senses through which to experience
the environment. The following steps of filtering, selection, rating and simplification have been incorporated into the
brain. Each of these artificial senses allows the agents to operate within and interact with their surroundings:
Read the ground: Because the agents are able to understand where they walk, this affects the way they interact with
their environment. We trained them to avoid streets, prefer to walk on the sidewalk but not to neglect streets for
crossing. In addition, we were even able to train the agents to have the ability to understand a whole set of colors.
This gives us the freedom to underlay our 3d model with a master plan readable to the agents.
Vision: As simplified replicas of humans, the agents are strongly controlled by their visual input. We use vision to
detect collisions, which is an overruling factor (Gibson, 1979). A change in color of an agent is used to
communicate its state to other agents. For example we express human emotions as a color code, allowing typical
emotions to be conveyed in situations such as when a collision occurs. Red, representing anger, or blue, representing
indifference, are visible in the agents.
Hear: As in reality, when we can detect the distant chiming of church bells, the ability to hear allows the agents
to detect stimuli, which are out of sight. Agents us the frequency and the amplitude of the noise to guide them
towards distant locations or points of interest which are out of sight.
4.2 Set of Goals
Several analytical tools have been presented which examine the correlation between how people move within, and
use, the urban environment (Penn and Turner, 2001). In our approach every agent has its own set of preferences in
relation to the specific places he wants to visit. For example the desire to stop for coffee in the morning on the way
to work may be common, but certainly is not representative for all commuters. In dynamic environments, such as
168
9
th

busy city streets at rush hour, the agent is able to adjust its path and goal accordingly. For example, the agent can
change its mind in real time. Each possible destination has a maximum occupancy which, when reached, reduces its
attraction for agents. Upon arrival, an agent can differentiate between buying a cup of coffee and having a whole
meal, and therefore spends time accordingly. The goals are set accordingly to the master plan, which consists of
function layers, such as living, working, industry etc., and creates an additional set of locations for special activities.
This is where the agents would typically spend time between work and home. By setting the master plan, every
urban planer has an idea where special activities are likely to take place. This would then either promote further
activity, or calm the surroundings. By compiling such information we reduce buildings to symbols of their function.
This allows us to focus on the elements that are frozen in the master plan, such as the width of the street and
sidewalk, the orientation of the doors and their size, and most importantly the use factor.
4.3 Output
During the simulation, every agent draws an individual color-coded line on the ground. The color change, triggered
by the brain, is representative of both his emotions and his analytical experience of the environment. Every agent is
entirely unique. The artificial intelligence conveys information about the effort it took the agent to reach the
designated points of interests, if his journey was too time consuming, and how much personal space was available to
him during this time. If a significant number of agents appear to be exhausted at the very same spot, we know that
we have to adjust something. Depending on the location a rest area with benches may be needed. Also, if the net
travel time appears to be too long we assume that the agents would rather take public or private transport then walk.
With this information we can adjust the stops of the public transportation system to increase the time efficiency for
the majority of occupants.
5. URBAN ENVIRONMENT
The workflow to create a virtual urban model for crowd simulation is similar to the workflow used by architects in
urban planning. Based on a number of maps (color-coded images and other rasterized data), we create street-
networks and rough volumetric models of the buildings. These volumes are used to distribute urban functions and
building density and guide the agent navigation as described in the previous section. Based on these rough building
volumes, we create the final detailed geometry needed for visualization and the low-polygon geometry used in
simulation. The original aspect of our workflow is the use of procedural modeling to automate the creation of street
networks and building geometries. A big advantage of this is the adaptability - the urban environment is able to
change for each simulation without redrawing from scratch. We automate the export of control data for the
simulation as well as the visualization, which differ radically. They both are stored inside the same procedural scene
description and will therefore always be in sync. The following sections describe our workflow in more detail.
5.1 Creating the Urban Layout

FIG. 2: Color-coded attribute maps are used to control the generation of street-networks and building geometries.
From left to right: topology, obstacles, height map and building heights (skyline).
We use a futuristic scenario in the area of Zurich, Switzerland, to describe our workflow. In this example, we use
several maps (see figure 2) to encode aspects of the urban layout (e.g. topography, elevation, obstacles, skyline, land
usage). To create the street network, we have two options: (1) generate a generic, procedural, rule-based network
which follows the attribute maps (e.g. along the color gradients) or (2) import a network from a vector data source
(e.g. from openstreetmap.org). For simplicity, we show a generic street network (see figure 2, left/middle). The
algorithm is an extension of the L-system based street network generator described in Parish and Mueller (2001).
169
9
th

The building plots can be created in a similar way: (1) by a generic subdivision algorithm or (2) by importing a
vector data source. Again, we show generic parcels (figure 3, right), also created with the algorithm described by
Parish and Mueller (2001). Once the building parcels have been generated, we are ready to apply the procedural
rules to create the building and street geometry.
5.2 Procedural Modeling using Shape Grammars
Most recently, research in architecture (and subsequently computer graphics) has produced a number of production
systems for architectural models, such as Semi-Thue processes, Chomsky grammars, graph grammars, shape
grammars, attributed grammars, L-systems or set grammars (Vanegas et al., 2009). All of these methods expose
different application possibilities and levels of efficiency to the user. The shape grammar concept has recently been
made more applicable to computer graphics and daily usage (Mueller et al., 2006) and is now commercially
available in the package CityEngine, which is used in this paper (Procedural Inc., http://www.procedural.com).
The key tools of the context-sensitive shape grammar as implemented in the CityEngine consist of: shape operations
for mass modeling, component splits for transition between mass and facade, split operations for building facades,
spatial query operations and more.
In this section, we show how the shape grammar, as implemented in the CityEngine, is used to model some simple
building volumes for the Zurich scene. Each building lot (parcel) is assigned a shape grammar rule set. A rule set
consist of production statements in the form:
Predecessor [case Condition1:] Successors 1 [case Condition2:] Successors 2 [else:] Successor N
With this the successors can be composed of several shape operations and query statements. Listing 1 contains a
small example rule set and figure 3 shows a possible result. Figure 4 shows the result of the application of the
example rule set to the complete scene.

// get building height from the skyline map
// intensity values of the red-channel are linearly mapped from
0..1 10..200
attr height = map_01(red, 10.0, 200.0)
// the first rule checks for rectangularity
Lot --> case geometry.isRectangular(10) :
case scope.sx-10 < scope.sz && scope.sx+10 > scope.sz:
RectQuad(height)
else: other rules
else: other rules
// scale the building and center it on the lot
RectQuad(h) --> s('.8,'1,'.8) center(xz) extrude("y", h) UShaped
// divide along the x axis into two wings
UShaped --> split(x){'rand(.3,.5): Facades | ~1: SideWings}
// subdivide again along the other horizontal axis
SideWings --> split(z){'rand(.3,.45): SideWing | ~1: NIL | 'rand(.3,.45): SideWing}
SideWing --> 30% : Facades
30% : split(x){ 'rand(0.2,0.8)
: Facades}
else : s('1,'rand(0.2,0.9),'1)
Facades
// the generation stops with the 'Facades' shape
Left: FIG. 3. A simple U-shaped mass model consisting of three volumes (shapes).
Right: Listing 1. This "CGA shape" source code produces simple U-shaped mass models consisting of three shapes.
Some parts have been omitted for simplicity.
170
9
th

FIG. 4: On the left, the resulting scene is shown with the same simple set of shape grammar rules applied to all
building lots. The figure on the right shows a close-up of the high-rise area in the center.
5.3 Generation of the Control Data for Crowd Simulation
In Section 4.1 discussed the input channels (e.g. ground colour, vision, sound) used by the agents in the crowd
simulation. In the current section, we describe how these channels are derived from the procedural urban model. In
other words: we need to extend our grammar rule set with an optional set of rules which trigger the generation of
input data for the simulation. Our crowd simulation setup needs three types of input:
- Simplified building and street geometry to visually guide the agents in order to prevent collisions.
- Locators for initial agent placement and points of interest to implement building functions.
- Terrain geometry with a texture map where additional color-coded control data is stored. For example, we are
using color intensity in the door areas to encode the number of people that are entering or leaving a certain
building.
FIG. 5: Illustration of the data-flow between the CityEngine (left) and the crowd simulation tool (right). Note that
all the necessary input data to the simulation has been generated automatically from a single grammar rule set.
The CityEngine contains an export algorithm which scans the names of the terminal shapes of the grammar
generation process (in the U-shaped building example above these were called 'Facades') and triggers the creation of
one of the input data types for the crowd simulation based on a set of name patterns. For a typical agent simulation
scenario the relevant terminal shapes are usually called 'door', 'window', 'sidewalk' etc. Figure 5 depicts the data-
flow between the CityEngine and the crowd simulation tool. The necessary modifications to an existing grammar
rule set look as follows:
1. Based on the simulation scenario and the tool (in our case Massive), choose the appropriate control
features (vision, sound, colours, vector fields, etc.) to trigger certain behaviours of the agent (walk, stop,
attract, etc.).
2. Identify the parts of the urban environment with which the crowd agents interact (e.g. doors, windows,
sidewalks etc)
171
9
th

3. Introduce rules, which trigger certain simulation features not necessarily visible when rendering the scene
(e.g. a locator on the bottom face of a building entrance). The main idea is to make these rules conditional
on a rule attribute to be able to comfortably switch between model generation for visualisation and model
generation for simulation.
4. Also introduce conditional rules to deactivate all geometry not needed in the crowd simulation.
For our final results, the previously mentioned modifications resulted in the following encodings:
Locators for agent sources: all terminal shapes called 'door' trigger the generation of a locator and a color
on the bottom-face of the doors. The color is used to encode the amount of agents emitted from this door.
Locators for points-of-interest: the doors also trigger a sound source, which is connected to it to attract
agents. We distributed 10 different frequencies for the ten different POIs (POI0,,POI9) according to the
master plan. If necessary, the distribution of the POIs can also be controlled by grammar rules.
Locators for background agent sources: the shape grammar splits the sidewalks into small stripes and
assigns agent sources to them according to a certain probability (see Figure 6).
FIG. 6: From left to right: (1) detailed geometry used for visualization. (2) The rough volumes used for agent vision
in the simulation. (3) The blue/red marked areas show the 'door' terminal shapes used to generate agent source
locators. (4) Volumes (gray), color-code (red) and locators (yellow) imported into the simulation tool ("Massive").
6. EXAMPLES
To prove our concept we are presenting two case studies. The first aims to push the ability of our method to the limit
by using a large number of buildings and many agents. With the second case study we use a higher level of detail.
6.1 Dbendorf
Dbendorf, a suburban centre of Zrich, currently houses a military airport, which will be made redundant. The area
in question is the biggest to be developed in Switzerland in the near future. With an increasing trend of living in
suburban areas there is increased pressure to develop the area into a compact new centre for Dbendorf and
surrounding areas. In this case we can show the density differences between the area of high-rise buildings in the
centre and the low-density living quarters on the outskirts. There is no absolute rule about density or traffic - for
some attractions an increased flow of occupants is favored, while for others this is not the case. We show in figure 7
the layer of personal space. The problem we identified here was that the entrance / exits of the buildings were
opposite to each other. They would be better placed in an asymmetric way to avoid collisions.
172
9
th

FIG. 7: Left: Quantified visualization of 6800 agents inside the given area. Middle and right: Occupant stress
analysis; a very intense magenta represents not enough personal space.
To handle the amount of agents in this case, we group agents into three major groups (Jager et al., 2001). The group
who are coming from home (GotoWork), the group who come from work (GotoHome) and the group who have no
specific goal to reach and occupy the streets as a present background noise.
6.2 Second Case
The setup consists of a pedestrian crossing with different typical activities, and a street, which represents an obstacle
to the agents. This time the agents find their way from their source to their final destination by making a selection
from a list of possible secondary goals and marking their way within their individual path. As a result we can see
increased traffic around the high-rise buildings as well as the crossing of the streets in the north (see figure 8). As
seen in the collision map, there is no stress problem at street crossing. However, even with the same amount of
pedestrians, there is a problem in the middle of the pedestrian walk way. This is due to several reasons. One reason
is the agents flocking behavior, where agents align themselves after increasing amount of iterations. Additionally, if
an agent is making decisions in that particular area it is likely that he will change his goals. Interestingly, the agents
do not favor the two asymmetric pedestrian crossings. In order to find the next crossing they can easily block each
others path. The total-time-spend map reveals the need for another public traffic stop at the south end to even out
the time needed to reach all goals.
FIG. 8: High-density crossing; from left to right: Density, collision map, total time spend to complete the list.
7. RESULTS
Our simulation method enables the urban planner to adjust the design from small scale to large scale. The high-
resolution pictures (figure 9) are also a method of communication to all stakeholders used to discuss ideas and
address problems. With an increased number of decisions made unilaterally, decision making time and meeting
requirements are reduced.

FIG. 9: High-density crossing; from left to right: Density, collision map, total time spend to complete the list.
173
9
th

With a limited amount of resources, this allows the urban planer to choose the smallest possible intervention to gain
the largest output. Small obvious changes are more likely to be accepted by the occupants and therefore have a
higher chance of creating a permanent effect on the city. Several owners adjust their house entrances after some time
to optimize their property, or alternatively a change in ownership occurs triggered by an increase in value and a need
for more area. In the design stage, these changes can be simulated and taken into account.

8. DISCUSSION
Even with useful results that are relatively easy to understand, we have to research possible deviations between our
prediction and reality. Despite reliable statistics, it is virtually impossible to reconstruct the decision process of
human beings. For this reason we concentrate on the path finding abilities of the agents and rely on predefined goals.
Another limitation of our method is the calculation time. The enhanced complexity of the urban layout increases the
calculation time exponentially.
9. FUTUREWORK
We have not yet been able to automate the input from the agents into our generic city model. It is possible to
totally automate this evaluation method with a combination of python scripting and visualization toolkits like
RenderMan from inside the CityEngine as well as from inside MassivePrime. Another step would be to adjust the
buildings and public transport networks to be autonomous and to be outfit with intelligence as well. The adapting
environment within our simulation breaches the current gap that exists because of the differences in time scale and
explains how sudden events form the city.
10. AKNOWLEDGMENT
We would like to thank the participants of the elective course Collaborative City Planning at the Chair for
Information Architecture, spring semester 2009, who designed the model of the case studies as well as Martina
Jacobi who did the preprocessing work for the simulation. Especially, I would like to thank Alice Vincent for her
tremendous involvement in the writing process.
11. REFERENCES
Arnott R. und Small K.A.: 1998, Urban Spatial Structure, journal of Economics Literature, Vol. XXXVI, Pages:
1426 - 1464
Gibson J J, 1979 The Ecological Approach to Visual Perception (Houghton Mifflin, Boston, MA)
Diener, R., Herzog, J., Meili, M.,de Meuron, P., Schmid, Ch, (2006). "Switzerland - An urban Portrait." ETH Studio
Basel: Conteporary City Institute, Pages: 64-109.
Jager, W., Popping, R. and van de Sande, H.: 2001, Clustering and Fighting in Two-party Crowds: Simulating the
Approach-avoidance Conflict, Journal of Artificial Societies and Social Simulation, 4(3).
LeBon, G.: 1895, Psychologie des Foules, Alcan, Paris.
Halatsch, J., Kunze, A., Burkhard, R., and Schmitt, G. (2008a). ETH Value Lab - A Framework For Managing
Large-Scale Urban Projects, 7th China Urban Housing Conference, Faculty of Architecture and Urban
Planning, Chongqing University, Chongqing.
Halatsch, J., Kunze, A., and Schmitt, G. (2008b). Using Shape Grammars for Master Planning, Third conference on
design computing and cognition (DCC08), Atlanta.
Magnenat-Thalmann, N. and Thalmann, D. (eds.): 2004, Handbook of virtual humans, Wiley, Chichester.
Mller P., Wonka P., Haegler S., Ulmer A. and Van Gool L. 2006. Procedural Modeling of Buildings. In
Proceedings of ACM SIGGRAPH 2006 / ACM Transactions on Graphics (TOG), ACM Press, Vol. 25, No.
3, pages 614-623.
Musse, S. R., Thalmann, D.: 2001, Hierarchical Model for Real Time Simulation of Virtual Human Crowds, IEEE
Trans. on Visualization & Computer Graphics, 7(2), pp. 152-164.
Parish Y. I. H. and Mller P. 2001. Procedural Modeling of Cities. In Proceedings of ACM SIGGRAPH 2001, ACM
Press / ACM SIGGRAPH, New York. E. Fiume (ed), COMPUTER GRAPHICS, Annual Conference Series,
ACM, pages 301-308.
Penn, A. and Turner, A.: 2001, Space Syntax Based Agent Simulation, in M. Schreckenberg, S. D. Sharma (eds.),
Pedestrian and Evacuation Dynamics, Springer-Verlag, Berlin.
174
9
th

Procedural Inc., Zurich, Switzerland; http://www.procedural.com
Still, G. K.: 2000, Crowd Dynamics, PhD thesis, Warwick University.
Aschwanden, G: 2008, Agent-Based Crowd Simulation for Urban Planning, In Proceedings of eCaaDe 2008.
Tecchia, F. and Chrysanthou, Y.: 2000, Real-Time Rendering of Densely Populated Urban Environments, in Proc.
Eurographics Rendering Workshop.
Vanegas, D. Aliaga, P. Mller, P. Waddell, B. Watson and P. Wonka. 2009. Modeling the Appearance and Behavior
of Urban Spaces. State of The Art Reports Eurographics 2009.

175
9
th

INTEGRATION OF AS-BUILT AND AS-DESIGNED MODELS FOR 3D
POSITIONING CONTROL AND 4D VISUALIZATION DURING
CONSTRUCTION
Xiong Liang, PhD Candidate
Department of Civil and Structural Engineering, Hong Kong Polytechnic University, Hong Kong, China;
07901081r@polyu.edu.hk
Ming Lu, Associate Professor,
Department of Civil and Structural Engineering, Hong Kong Polytechnic University, Hong Kong, China;
cemlu@polyu.edu.hk
Jian-Ping Zhang, Professor,
Department of Civil Engineering, Tsinghua University, Beijing, China;
zhangjp@tsinghua.edu.cn
ABSTRACT: This paper describes an on-going research on as-built and as-designed model integration and
visualization to support critical physical tasks during construction such as positioning components during
installation and inspection of the geometric dimensions, the layout and the orientation on as-built components. The
objective of this research is to develop a generic, extensible framework to support the above integration and
visualization processes. Critical issues, principles and challenges arising from the development of the framework are
discussed. Some preliminary results are presented in a case study to demonstrate the usefulness of the framework in
terms of leveraging data collection technologies and visualization technologies to improve the current construction
practices. The proposed framework is supposed to provide a cornerstone for developing the next generation 4D
system, which will be used not only for project management tasks but also for physical tasks during construction
engineering operations.
KEYWORDS: Visualization, As-built and As-designed Integration, 4D CAD, Construction tasks.
1. INTRODUCTION
The 4D technology and its application in architecture, engineering, and construction (AEC) have been intensively
studied in the areas of construction planning (Liston et al. 1998), constructability review (Hartmann and Fischer
2007) and site layout planning (Ma el. 2005) etc. 4D technology has improved many practices in the construction
industry. For example the construction planning is improved through visualizing the animated 3D construction
sequence which leads to better communication among stakeholders involved, enhanced visibility and clarity of
project or process plans, more dynamic and integrated review of different plan options.
However, to date, the 4D technology has not yet been accepted on a large scale in construction practices. There are
many reasons to account for such hesitation: The complexity of 4D modeling and the lacks of integrated tools require
talents with special skills (Haque and Mishra 2007); Barrett (2000) attributed it to the dynamic, fragmented and
combative nature of the construction industry. In our past experiences on 4D application in construction projects in
China, the lack of tangible support to contractors critical tasks is identified as an important factor that has hindered
practitioners to adopt the 4D technology during construction execution. We enlisted many project managers views
on 4D technology, one common opinion is that the cost of 4D modeling is very high, however, it is only used on one
or two occasions in the early stages, e.g. to facilitate the communication of contractors bid proposals with owners.
Then the 4D models are shelved in the cabinet without follow up applications. This observation is also echoed by
Hartman et al. (2008) who analyzed twenty-six case studies of 3D/4D model applications on construction projects.
They discovered that practitioners mostly used the models in one application area only. They suggested that further
177
9
th

research on the integration of 3D/4D model technologies into work and business processes of project teams should
address the more widespread use of 3D/4D models throughout the lifecycle of a project.
At present, research of 4D technology and its application largely focus on management tasks such as presentation
and communication of project plans and construction methods. There is little research reporting the use of 4D models
during the construction execution phase to support physical tasks like positioning components during installation and
inspection of the geometric dimensions, the layout and the orientation on as-built components. Bernold (2002) also
stated that modern design software is able to digitally model building elements in their spatially correct
configurations, and without interference. However, when it comes to the setting of actual construction in the field,
those spatial models have not yet found much value-added applications.
On the other hand, reality capture technologies have matured sufficiently to be utilized for as-built data collection on
construction sites. Many advanced positioning and data collection technologies (GPS, total station, laser scanner) are
being gradually adopted by construction practitioners. The feasibility and suitability of combining these advanced
technologies with computer-based visualization technology to improve current practices in the construction industry
have been studied by many researchers. Bernold (2002) described their work on digitally merging spatial design data
with the digital model of equipment working on implementing the design in support of site operation. Akinci et al.
(2006) combined automated data capture technologies (laser scanner, temperature sensor) with designed project
models for quality control on construction sites. In recent years, augment reality (AR) which superimposes computer
generated images over a humans view of the real world has been studied to demonstrate its feasibility and to
validate its suitability in the construction industry (Shin and Dunston 2008).
Despite the fact that there are many benefits to be materialized from the integration of as-built and as-designed
models in construction, the lack of easy-to-use and integrated tools has hindered the application of the above
mentioned technologies in the construction industry. In this paper we provide a framework for developing the next
generation 4D system that facilitates automated as-built and as-designed model integration and visualization, such
that: (1) the efforts on 4D modelling can be fully utilized throughout the whole life cycle of construction project; in
particular, the physical tasks in a construction project can be improved in terms of quality and efficiency; (2) The
management tasks can be directly linked with the physical tasks so as to enhance project management and control.
We also developed a prototype system to demonstrate the application of the framework to practical construction
scenarios.
2. DESCRIPTION OF THE FRAMEWORK
Shin and Dunston (2008) claimed that a construction process involves various tasks that deal with the creation of
physical structures and elements and most often require visual information to understand and communicate their
complexity and relationship to existing structures or elements. They indentified eight work tasks (layout, excavation,
positioning, inspection, coordination, supervision commenting, and strategizing) may potentially benefit from AR
support. Meanwhile 4D CAD decomposes 3D product models into building components which are linked with
construction activities, some of which can be mapped to the above work tasks. This provides an opportunity to use
the 4D technology to support some daily physical tasks on construction projects if the 4D model is integrated with
real-time site data. This extension to current 4D CAD systems will in turn drive the construction industry to adopt
the 4D technology. However, the modeling and application procedures should be simplified such that construction
engineers would fully leverage the extended 4D tool to cope with construction site tasks.
Fig. 1 illustrates the concept of as-built and as-designed integration and visualization based on the proposed next-
generation 4D technology. In this framework, some critical issues need to be addressed: Firstly, the available as-built
data that may be useful for supporting construction tasks should be indentified and assessed. Secondly, the
decomposition and organization of 3D product models should be considered during 4D modeling processes at
different levels of detail. The next generation 4D system is supposed to be used in both management tasks and
physical tasks throughout the lifecycle of a construction project, so different requirements of the two types of tasks
need to be addressed carefully. Thirdly, the as-built data and the as-designed models need to be correctly registered
in the 3D graphical engine: for example, the two local coordinate systems need to be aligned; The as-built data need
178
9
th

to be transformed to meaningful representation in 3D in order to support physical tasks. Lastly, different data
collection technologies have different data structures and different data exchange protocols, which should be
synthesized in an agile way to make this framework extensible and scalable. In the following sections, the above
critical issues are described in detail, and a case study of implementing the framework to support physical tasks
based on a prototype system are presented.

FIG. 1: Overview of the as-built and as-designed integration and visualization framework
3. TAXONOMY OF AS-BUILT DATA
Researchers have studied many types of as-built data for a wide range of application areas in construction, e.g., RFID
and GPS positioning data for resource and material tracking (Ergen et al. 2007; Song et al. 2006); Laser scanner and
temperature sensor data for construction quality control (Akinci et al. 2006); Camera data for enhancing situation
awareness (Katz et al. 2008). It is important to understand what kind of data is suitable for supporting a particular
physical task through visualization. Kiziltas et al. (2008) assessed field data capture technologies for construction
and facility/infrastructure management. Shin and Dunston (2008) identified eight suitable application areas of
visualization technologies in industrial construction. Based on their work, three types of as-built data are selected
that hold promises for supporting construction physical tasks through as-designed and as-built integration and
visualization. The three types of as-built data, their related data collection methods and supporting physical tasks are
listed in table 1.

FIG. 2: Current positioning practices during the installation of steel column
179
9
th

TABLE 1: Suitable as-built data types in terms of visualization
As-built Data Data collection method Supporting physical tasks
Point GPS, Total station Positioning, Marking
Point clouds Laser scanners Inspection
Image Camera, Video camera Positioning, Inspection
A point is represented by three Cartesian coordinates X,Y and Z, which is the most important data collected from
todays surveying practices in a construction project. It is widely used in positioning a target location on a
construction site. Shin and Dunston (2008) indentified that when a positioned element does not require a highly
accurate location, the conventional visual information (marked points or lines) is clear and adequate. However, when
an element needs to be positioned with high accuracy, a marked target location may not be enough. For example,
some elements like steel columns require orientation control during positioning, but it is sometimes hard to indicate
an accurate 3D orientation for a large element in space only with limited target points. In this case, the element being
installed is positioned based on checking a number of reference points or lines, and adjustments to errors on the
position and orientation of the element are made. Workers continuously check and adjust the position and orientation
until the element is placed as per designed. It is a time-consuming loop procedure: fix the element temporarily, check
its location with a measuring device, adjust it, and then check it again. Fig. 2 illustrates the above critical process for
installation of a steel column on a high-rise building. It is obvious that visualizing the points data measured in
relation to the as-designed 3D model will improve the installation process in terms of quality and efficiency.
Point cloud is a set of vertices in a three-dimensional coordinate system, which are generally created by 3D laser
scanners. The point clouds are converted to triangle mesh models, non-uniform rational basis spline (NURBS)
surface models, or CAD models through a process commonly referred to as reverse engineering. Akinci et al. (2006)
studied the feasibility of integrating the point clouds data with a project model for quality control. Raw laser scanned
data contain various types of noises resulting from site constraints. In order to reconstruct the as-built model, the
noises need to be filtered. Furthermore, the point clouds from each scan are represented in the local coordinate
system of the scanner, which makes it impossible to perform direct comparisons with the as-designed model for
defect detection and analysis. Therefore, it is necessary to align all of the scans in a common coordinate system -a
process known as registration. Enabling automatic registration is one challenge that will be addressed in the proposed
4D framework.
An image represents a snapshot of the real world view human sees through a camera, a video camera or a head
mounted display (HMD) device etc. The superimposition of a designed 3D model on the image generates a new
scene that human cant see directly in the real world, for example, user could see the column that has not yet been
installed as if it were already installed. This computer graphic technology is named as augmented reality(AR). The
AR technology is useful for supporting construction physical tasks. However, the accurate tracking of users
viewpoint and orientation is a bottleneck to AR applications due to the lack of tracking technologies that are
functional at the construction site. Behzadan and Kamat (2007) investigated the application of the global positioning
system and 3 degree-of-freedom angular tracking gyros to get the viewpoint and orientation information needed for
accurate registration. Although their method had some limitations such as the accumulation of drift error and errors
inherent in GPS positioning, their work proved the feasibility of using AR technology for supporting the
visualization of construction simulation, and making the simulation more realistic.
4. DECOMPOSITION AND AGGREGATION
4D technology decomposes the product 3D model into components which are linked with construction activities in
order to produce construction process 3D animation. The decomposition mechanism is closely related to the
construction planning, however, the level of detail depends on the planning purpose. When a project manager uses
the 4D tools for project planning, they may not create a detailed schedule for handling each production component.
However, in order to extend the use of a 4D model into production tasks during construction, the decomposition of a
180
9
th

product model into the proper production level of detail is very important. For example, in Fig. 3 the 3D model on
the left hand side is a steel component of a high-rise building. It depicts the decomposition of the design-centric
product model prepared by the designer. While the three sub-components on the right hand side represent the
decomposition of the production-centric product model prepared by the contractor.
Different from previous applications of 4D technology in construction planning, this research entails more detailed
decomposition of 3D product models for handling construction physical tasks. As shown in the left part of Fig. 3 a
steel column segment of a high-rise building is treated as one component when the planner makes a rough
construction schedule. However, during the installation, the component needs to be divided into three parts due to the
limited lifting capacity of the cranes (construction method constraint). The 4D modelers should consider this
requirement and the construction method if the model is intended to be used for supporting the construction tasks in
the later construction stage.
According to Hartmann et al. (2008), previous experiences of applying 3D/4D models across different project phases
showed that a large amount of investments into the application of 3D/4D models in later stages in the project
lifecycles could have been avoided if the modelers and project managers had created earlier models with the
intention for later use on the project. However, the 3D models used for different application purposes by various
stakeholders require different levels of detail. The current 4D models for project planning are not detailed enough for
representing physical tasks during construction. For example, Fig. 4 shows that the coordinates for two prisms and
two tags are needed for installation control, however, the 3D as-designed model for 4D planning commonly does not
contain the information of these reference points needed for controlling of position and orientation during
installation.

FIG. 3: Decomposing the model according to construction method

FIG. 4: Design model VS. Production model
During the early stage like bidding stage, it is very difficult for 4D modelers to foresee the above mentioned data as
needed for the later applications, because construction is dynamic and fragmented in nature and different parties have
their own priorities and interests. The 4D system needs to provide interface to input various types of information
by different stakeholders in different stages. One aspect of the important information for handling physical tasks is
related to the control points (fiducials or marks) demonstrated in Fig. 4, which should be aggregated into the design
models associated with the later stages of construction, i.e., the detailing, manufacturing and installation. From the
181
9
th

perspective of 4D system development, the system designer should consider how to represent and organize the
building components into objects by using object oriented programming languages. The composite pattern is suitable
to deal with this design problem. Because this pattern ignores the difference between compositions of objects and
individual objects. If this pattern is used to design the structure of objects that represent the building components in
Fig. 3, a 4D modeler can use the same function to manipulate the composite component as well as the three sub-
components.
5. REGISTRATION AND TRANSFORMATION
A variety of data need to be communicated between the system that is responsible for as-built data collection and the
4D system: the geometric data, the registration data, the attribute data indentifying the type of as-built data, and the
time of measurement etc. In order to realize an extensible framework that supports multiple data collection devices,
we designed a two level communication protocol, in which the common data that is identical for all kinds of data
collection devices i.e., the time of measurement, the position and orientation of the devices viewpoint, and the
identifier of the data type etc. is processed at the first level. Then we used the identifier to indicate the type of as-
built object created in the 4D software system, which in turn would communicate with the data collection system to
receive the data at the second level.
After the as-built data are transferred into the 4D system, it should be registered in the 3D graphic engine before
being integrated with as-designed models. There are a variety of registration processes and methods for different data
collection technologies. The registration of a point is relatively simpler than the laser scanner generated point cloud.
For point registration, it just needs some geometric transformation between a total stations local coordinate system
and the as-designed coordinate system. But for the raw laser scanned point cloud, it contains many sources of noise,
each requiring a filter for removal. Furthermore, at a single position, a laser scanner can only capture points on an
object in its line of sight. As a result, the occluded parts of the object need to be scanned from other directions. This
requires a process of aligning several scans into a more comprehensive data set (Kiziltas et al. 2008). The diversity of
the registration requires an extensible application framework that allows the registration of various types of data
without manipulating the complex 3D graphic engine.
There are four types of established methods for registration of collected 3D site data (Akinci et al. 2006). The most
important requirement for registration of the site points data is to capture the total stations position. Considering a
practical construction project, the site usually has fixed markers, also known as fiducials, the position of which are
known for the purposes of site surveying and layout. Those markers are also recognized in the 4D designed model
and are used to aid the registration. If the total station is placed at a known position on construction site, its easy to
align the real world coordinate system and the as-designed model coordinate system during registration. However,
when the total stations position is unknown, its coordinates can be calculated as follows:
(1)
where , and are the coordinates of the point , which are known in advance from the 4D model, and
is the distance between the point and the total station, which can be readily measured by the total station.
When the position of the total station is registered, the surveyed control points on a building component can be
transformed to the as-designed coordinate system. The total station should measure at least three marked points on a
building component in one surveying cycle. One of these points is used to calculate the translation transformation,
which yields a vector: The designed coordinates of the three points denoted
by are known in advance, and the as-built points are surveyed by the total station at a
particular moment of time denoted by , which are then adjusted by the transformation
vector , yielding three new points denoted by . Then the rotation matrix
can be calculated as follows:
(2)
182
9
th

is a matrix. Thus, applying the vector and rotation matrix to the as-designed 3D model of the
building component will create a new 3D model to represent the as-built position and orientation of the component
being installed at that time. The transformation vector and rotation matrix can be combined in a transformation
matrix. The concept of this integration and visualization on a case study for installation positioning is demonstrated
in the following implementation section.
6. PROTOTYPING AND IMPLEMENTATION OF THE FRAMEWORK
To validate the feasibility of the framework, a prototype system was developed based on an in-house 4D system
named as 4D graphics for construction and site utilization (acronym 4D-GCPSU) Version 2009. Readers may refer
to Chau et al. (2005) and Wang et al. (2004) for detailed information of its earlier versions. In this prototype system,
the as-designed model is integrated with the as-built data collected by the total station to visualize the position and
orientation deviation of the building component being installed.
The first step is to link the production unit with a position-tracking system which will control the total station to
survey the marked points on a construction unit automatically. To do this, the user selects the construction unit from
the visualization screen or from the WBS tree structure via mouse clicks (see the GUI steps tagged with 1 and 2 in
Fig. 5). The selected production unit is highlighted with a bounding box and also shown in a small window at the
lower left corner. Then by clicking the tracking button (see the GUI step tagged with 3 in Fig. 5), a corresponding
entity will be created (see the GUI step tagged with 4 in Fig. 5) to control the visualization of the as-built position
and orientation of the construction unit in the 3D graphic engine during the installation-tracking process.

FIG. 5: Graphical user interface of the prototype system

FIG. 6: Current practice on positioning the steel columns
183
9
th

The case study in Fig. 5 is based on construction of a skyscraper of 103 storeys. It is a composite structure,
comprising a reinforced concrete core tube integrated with the perimeter diagrid frame to provide overall structural
stability. The perimeter diagrid frame has doubly curved elevations in its design such that it is a challenge to
accurately determine the position and orientation of the steel segment being installed. In the current practices, the
contractor uses a total station to measure the geometric center of a diagrid frame segment shown in Fig. 6a.
However, because at least three points are needed to determine the orientation of a solid object, only surveying one
or two points may lead to some orientation errors which cannot be easily detected in the current practice. For
example, in Fig. 6b only measuring point o cannot detect the triangles rotation about this point. Because this kind of
rotation will not change the coordinates of the point o. Similarly, only measuring two points (o and p in Fig. 6b)
cannot fix the triangles rotation about the line connecting the two points in 3D space.
Furthermore, in current practice the site crew needs to search the designed coordinates during installation
positioning. Significant time in a crews shift is actually spent in obtaining and understanding design information. It
is clear that more convenient, timely access to this information would considerably benefits task performance (Wang
and Dunston 2006). In the implementation of the framework, we propose the integration of the designed coordinates
of control points (Fig. 7a) with the real-time data of control point positions collected by a total station in a 4D system
for visualization, which improves the current error-prone approaches and also minimizes the crews time spent in
shift between checking design and adjusting installation (Fig. 2).

FIG. 7: The integrated automatic positioning process during installation

FIG. 8: The visualization of the installation process and the orientation error
184
9
th

In this framework, two control points are marked on the steel components using reflective tapes in addition to the
two prisms fixed on the geometric center points, one point for determining position and the other three points for
determining orientation (Fig. 7a). The coordinates of these control points as per design for each construction unit are
inputted into the 4D model before installation activities start. So the site crew doesnt need to search the designed
coordinates during installation. When the installation activity starts, the control points are tracked in real-time by the
total station which will automatically survey each point in one cycle (Fig. 7b). When one surveying cycle is finished,
the coordinates of these points will be transferred and registered into the 4D system. After finishing the registration,
the position and orientation deviation between the as-designed 3D model and as-built component can be calculated,
which yields a transformation matrix including translation and rotation. Applying the transformation matrix to the
3D as-designed model generates a new 3D model to visualize the real position and orientation of the component
being installed, which is rendered in a different colour: the white colour model represents the as-designed
component, and the gray colour model represents the as-built position and orientation of the components being
installed at that time in Fig. 7c.
In Fig. 8a, six snapshots of the integrated visualization of the lifting process are presented in the form of filmstrip to
show the sequence by which the component is gradually approaching its designed position and orientation. When it
is exactly positioned on site the as-designed and as-built models will converge as shown in the last snapshot. Aided
by integration and visualization of as-built and as-designed data, the site crew could easily detect any derivation and
also can get real-time feedback from any installation adjustment operation. This will then further guide the crew on
how to operate the hydraulic jacks or cranes. Fig. 8b zooms in one snapshot which demonstrates a rotation error
around the geometric central point. In the current practice, by surveying the central point alone this error cannot be
determined. This proposed approach to construction-surveying integration and 4D visualization potentially enhances
the efficiency and quality of the installation of steel components.
As-designed and as-built model integration and visualization improve current construction practices in regard to the
tasks of positioning control, guidance of installation, and quality inspection etc. This paper has presented a
framework to integrate and visualize the as-designed and as-built models in a 4D CAD system which extends the
current 4D technology from supporting management tasks to guiding physical tasks. The critical issues on
developing the framework are discussed; the feasibility of the framework is demonstrated with a case study of
structural steel component installation in high-rise building construction. Note that Milgram and Colquhoun (1999)
defined Mixed Reality as including the full range of combinations of virtual and real environment. In this case, we
applied the points data collected on site to visualize the position and orientation of as-designed CAD object which
can be regarded as being close to the virtual environment. Thus, the application presented here can be suitably
categorized into augmented virtuality (AV).
The framework can be further extended to accommodate three promising types of data collected from construction
sites, namely, reference points on a production unit being installed, point clouds for representing the actual building
product, and the image of actual building product taken by cameras. The other two types of data will be addressed in
the authors ongoing research, aimed to study their feasibility on supporting construction physical tasks, and to
validate the extensibility of the framework.
8. ACHNOWLEDGEMENTS
The research presented in this paper was substantially funded by a National Key Technology R&D Program in the
11
th
Five Year Plan of China (No. 2007BAF23B02) and a Hong Kong Polytechnic University Niche Area Research
Grant (A/C No. BB89). The writers also thank the West Tower Contractor Guangzhou Construction Group and
China State Construction Engineering Corp. for cooperation and assistance in conducting the research.
185
9
th

9. REFERENCES
Akinci, B., Boukamp, F., Gordon, C., Huber, D., Lyons, C., and Park, K. (2006). "A formalism for utilization of
sensor systems and integrated project models for active construction quality control." Automation in
Construction, 15(2), 124-138.
Barrett, P. (2000). "Construction management pull for 4D CAD." Proceeding, Construction Congress Vi, ASCE,
Reston, Va., 977-983.
Bernold, L. E. (2002). "Spatial integration in construction." Journal of Construction Engineering and Management-
Asce, 128(5), 400-408.
Behzadan, A. H., and Kamat, V. R. (2007). "Georeferenced registration of construction graphics in mobile outdoor
augmented reality." Journal of Computing in Civil Engineering, 21(4), 247-258.
Chau, K. W., Anson, M., and Zhang, J. P. (2005). "4D dynamic construction management and visualization
software: 1. Development." Automation in Construction, 14(4), 512-524.
Ergen, E., Akinci, B., and Sacks, R. (2007). "Tracking and locating components in a precast storage yard utilizing
radio frequency identification technology and GPS." Automation in Construction, 16(3), 354-367.
Haque, M. E., and Mishra, R. (2007). "5D virtual constructions: Designer/constructor's perspective." Proceedings of
10th International Conference on Computer and Information Technology (Iccit 2007), 134-137
Hartmann, T., and Fischer, M. (2007). "Supporting the constructability review with 3D/4D models." Building
Research and Information, 35(1), 70-80.
Hartmann, T., Gao, J., and Fischer, M. (2008). "Areas of application for 3D and 4D models on construction
projects." Journal of Construction Engineering and Management-Asce, 134(10), 776-785.
Katz, I., Saidi, K., and Lytle, A. (2008). "The role of camera networks in construction automation." 25th
International Symposium on Automation and Robotics in Construction - Isarc-2008, 324-329.
Kiziltas, S., Akinci, B., Ergen, E., Tang, P., and Gordon, C. (2008). "Technological assessment and process
implications of field data capture technologies for construction and facility/infrastructure management."
Journal of Information Technology in Construction 13, 134-154.
Liston, K. M., Fischer, M., and Kunz, J. (1998). "4D annotator: A visual decision support tool for construction
planners." Computing in Civil Engineering, 330-341.
Ma, Z. Y., Shen, Q. P., and Zhang, J. P. (2005). "Application of 4D for dynamic site layout and management of
construction projects." Automation in Construction, 14(3), 369-381.
Milgram, P., and Jr., H. C. (1999). "A taxonomy of real and virtual world display integration." Mixed Reality:
Merging Real and Virtual Worlds, Ohmsha Ltd, Heidelberg, Germany, 5-30.
Shin, D. H., and Dunston, P. S. (2008). "Identification of application areas for Augmented Reality in industrial
construction based on technology suitability." Automation in Construction, 17(7), 882-894.
Song, J., Haas, C. T., Caldas, C., Ergen, E., and Akinci, B. (2006). "Automating the task of tracking the delivery and
receipt of fabricated pipe spools in industrial projects." Automation in Construction, 15(2), 166-177.
Wang, H. J., Zhang, J. P., Chau, K. W., and Anson, M. (2004). "4D dynamic management for construction planning
and resource utilization." Automation in Construction, 13(5), 575-589.
Wang, X. Y., and Dunston, P. S. (2006). "Compatibility issues in augmented reality systems for AEC: An
experimental prototype study." Automation in Construction, 15(3), 314-326.
186
9
th

AUGMENTING SITE PHOTOS WITH 3D AS-BUILT TUNNEL MODELS
FOR CONSTRUCTION PROGRESS VISUALIZATION
Ming Fung SIU, BEng,
Dept. of Civil and Structural Engineering
Hong Kong Polytechnic University;
06040959d@polyu.edu.hk
Ming LU, Associate Professor,
Dept. of Civil and Structural Engineering
Hong Kong Polytechnic University;
cemlu@polyu.edu.hk
ABSTRACT: This paper describes an Augmented Reality (AR) application by superimposing computer generated 3-
dimensional graphics over real world photos. Current practice in construction uses separate drawings to represent
the as-designed and as-built structural components, which remains a less effective means of communication.
Applying AR technology has been proposed to remedy this situation but it has yet to exert broad impact on
construction management. The reason is that there exists no accurate, simple and cost effective approach to
implementing this technology. New analytical methods resulting from recent research greatly reduce the effort and
complexity in applying AR. This paper presents a case study of visualizing micro-tunneling construction progress by
applying a new analytical method for augmenting site photos with 3D building product models and contrasts the new
method against "virtual camera" function provided by commercial software. The analytical approach is found to be
accurate and easy-to-apply. This study reveals the possibility of developing an automated solution based on the
analytical method of augmenting a series of time-lapse site photos with 3D as-built models, potentially benefiting
practitioners in visualizing construction progress of subsurface pipeline infrastructure.
KEYWORDS: Augmented reality, Site photo, Construction progress, Visualization, Automation.
1. INTRODUCTION
Graphical simulation by augmented reality (AR) technology provides a useful tool for evaluating the building
product against its design and investigating the constructability of construction activities during planning and
construction stages. AR technology has made headways in recent years, driving the acceptance of AR by researchers
and practitioners. For instance, comprehensive AR applications in construction were evaluated by Shin and Dunston
(2008). Several experiments were carried out for illustrating the feasibility of applying AR such as superimposing a
CAD model of an excavator over a real site background in operations simulation (Behzadan and Kamat 2007).
In previous attempts of applying AR technology, the tracking system relied on GPS and three-degree-of-freedom
orientation tracker to determine the position and orientation of cameras viewpoint. However, the system has yet to
provide speed and accuracy for achieving real-time tracking (Kensek and Noble 2000). In the development of AR
technology, overlapping view frustums of both virtual and real photos results in augmented photos. Generally,
manual manipulation is needed in fine-tuning the position of the model; as a result, the process is time consuming
and the AR modeling effort is hard to repeat. For instance, Kim and Kano (2008) conducted an experiment to
illustrate the difference between the as-built products and as-design drawings by superimposing actual construction
photos and virtual reality (VR) images. However, manual adjustment was needed to align and superimpose the
virtual and actual scenes. Thus, it raises the present research interest in employing AR by using a camera only and
processing the superimposition analytically and automatically.
An analytical approach to augmenting site photos with the as-build bored pile model has been proposed based on the
photogrammetry theories by Dai and Lu (2009). It has not been attempted yet to visualize a completely underground
feature and compare the new analytical approach with the manual approach. To enable engineers or clients to have
187
9
th

an integral view of the actual construction site situation and process, dynamic visualization of a pipeline tunnel under
construction will also be presented in this paper. In short, augmented construction photos will be valuable in
documentation, progress controlling and construction monitoring in practice. This paper focuses on the comparison
between the new analytical approach and the manual approach. It also discusses the potential of developing an
automated solution based on the analytical technique and the value added to practice with a case study of visualizing
micro-tunneling progress.
2. RESEARCH PROBLEM DEFINITION
In the practical micro-tunneling site, a site location plan (2D CAD drawing) was generally employed for presenting
the location, layout and alignment of the structure. As for the project, a method statement indicated that "a pre-
condition survey with sufficient site photos showing the ground condition, existing vegetation and plants etc. is
required. " However, there was no effective way to express the construction site situation together with the proposed
tunnel design and to illustrate the progress of on-going pipe jacking operations. In this case, only laser was used in
controlling the alignment of the tunnel. The resulting survey records on the alignment control for pipe jacking were
submitted to the client. Piles of data sheets made the recording and communication difficult and confusing. Hence, it
is worthy of investigating the potential in generating augmented photos to visualize the as-designed underground
structure alongside the as-built progress of the tunneling project in dynamic views, and in a cost effective way.
The aim of this study is to verify the potential for visualizing 3D as-designed or as-built models with real site photos
by applying an analytical photo-augmentation approach. The research is also intended to generate a series of
dynamic views of tunneling construction by using as-built data analogous to time-lapse video keeping. In conducting
the case study, the new analytical technique and the traditional approach are contrasted, and the potential of
automating the newly developed analytical approach along with site constraints and limitations are evaluated.
3. ANALYTICAL METHOD VS MANUAL MATHOD
The new analytical method is based on photogrammetric theory (Dai and Lu 2009). Photogrammetry is a science or
art of obtaining reliable measurements by means of photographs (Thomson, et al. 1966). The method is briefly
introduced as follows: by defining two known points, a cameras perspective location O ( ) and the
objects focus location ( ), a normal vector N is introduced by linking the two points as illustrated in
Fig. 1a. It is a vector perpendicular to the image plane of the camera, which extends from the perspective center of
the camera to the focal point of the photo. Three orientation angles regarding the orientation of camera can be
computed by using the normal vector. These three angles control the orientation of the camera. Fig. 1a and Fig. 1b
show clearly the orientation of the perspective center of the camera with respect to the object coordinate system.

FIG. 1a: Orientation angles FIG. 1b: Orientation angle s
188
9
th

With the Normal Vector N, the three orientation angles ( ) defining the orientation of camera are computed by
Eq. (2) and Eq. (3).
(1)
(2)
(3)
Regarding the rotation of in the Fig. 1b, with s measured from the positive y-axis to the
photogrammetric nadir point. By defining to be , . This implies the orientation of the camera in taking
picture is always in the "landscape" mode.

FIG. 2a: Original orientation of the camera FIG. 2b: Rotate about axis

FIG. 2c: Rotate about axis FIG. 2d: Rotate about
In taking photos, the camera is only allowed to rotate and angles about axis and axis respectively, as
shown in Fig. 2a and Fig. 2b. Rotation about the normal vector N (Fig. 2d) is prohibited to ensure the camera taking
photos in "landscape" mode.
With the three orientation angles, the relationship between nth target points in the object space to the corresponding
point in the image plane can be established by the Collinearity equation as shown in Eq. (4).
189
9
th

That is,
(4)
where
(5)
and
(6)
Another form of Collinearity equations with the scaling factor being eliminated is given in Eq. (7) and Eq. (8).
In these equations, c is the focal length of the camera, ( ) is the coordinates in the image coordinate system and
( ) is the coordinates in the object coordinate system. By substituting a known points coordinates in the
object space into the equation, the corresponded image coordinates can be calculated.
(7)
(8)
That is, ( ) = (9)
Eq. (9) implies that the image coordinate is a function of object coordinate. Thereby, through establishing a
mathematical regular grid platform the two photos can be superimposed analytically and precisely. The grid can be
achieved by using either the CCD/CMOS size parameters or the "extra point" method, as are described below.
The image coordinate plane (or termed as the photo coordinate plane) in the camera lies on the Charge-Couple
Device (CCD) or the Complementary Metal Oxide Semiconductor (CMOS) with different camera configurations.
They are image sensors with different technologies for capturing images digitally (DALSA Corporation, 2009)
while their mechanisms are similar in capturing an image. An image is projected through a lens onto the photoactive
region of CCD or CMOS, then, the photo being captured is digitized and stored in some form of memory processed
by the controlling circuit. The size of the image coordinate system is defined by the size of CCD or CMOS. Hence, a
mathematical regular grid platform can be established from the dimensional parameters of CCD/CMOS for
superimposition.
The CCD/CMOS technique proposed provides a means to fit a real photo into the definite size of the frame. The size
of the frame is controlled by the dimensions of CCD or CMOS, with the focal point being the centre of the photo.
The image coordinates of each point in the real photo can be evaluated on this grid. By using the image coordinates
190
9
th

of one more point from the object coordinate system, in addition to the focal point in the virtual photo, a virtual
photo can be superimposed with the associated actual photo precisely.
The "extra point" method is used if the size of CCD/CMOS is not known. One extra point is needed in the actual
photo in order to set up the grid together with the focus point, which is similar to the virtual photo processing being
discussed. Then the two photos can be superimposed by using the grid precisely.
In regard to the manual method, some commercial software, such as Autodesk 3ds Max Free 30-day trail software
(Autodesk Inc. 2009), a photo being captured by a real camera can be inserted as environmental map (specifically,
camera map) to render a background scene for a virtual camera. Then, augmented photos result in this way.
The use of the "virtual camera" concept is common in commercial of 3D graphics. Theoretically, if the focal length
of the lens of the virtual camera and the real camera are the same, given the identical position and orientation of the
camera, the scale of the images in the photo is fixed. However, the dimension and aspect ratio of photos vary with
cameras of different models and brands. Resizing, cropping and rescaling need to be done manually before the
photos can be superimposed correctly. Notwithstanding manual operation is feasible, it is not a quick and convenient
approach in applying the proposed AR technology in construction. Automation by the use of analytical method will
of benefit in terms of accuracy, time and cost.
In contrast with the traditional superimposing technique, the newly developed analytical approach accurately
calculates the image coordinates in both real and virtual photos. These virtual vs. actual photos are seamlessly
mapped together on the coordinate platform, thus outperforming the manual method in terms of accuracy and
automation potential.
4. CASE STUDY
A real case study was conducted to compare the analytical approach and the manual approach. It was micro-
tunneling construction by jacking concrete sleeve pipes across the So Kwun Wat Nullah in Hong Kong, consisting
of pit construction and installation of 1200mm internal diameter precast concrete sleeve pipes by pipe jacking
method as shown in Fig. 3. Two parallel micro-tunnels were planned to be constructed. When the case was
conducted, the construction of the micro-tunnel for electrical cable had been completed and the other was in the
planning stage.

FIG. 3: Jacking pit and concrete sleeve pipe
4.1 Site Constraints
Before conducting real site experiment, site investigation was carried out to determine the possible locations of
setting up the camera station and fixing the focus point in the context of site constraints. Through the investigation,
it was found that identifying a location with a view showing the whole construction site and covering the whole
tunnel was difficult. The reason was the construction site was relatively large. The focus point could not be chosen
at the centre of the Nullah since the coordinates of the focus point could not be determined by surveying
instruments. Thus, the location of the focus point should be near the centre of the construction site where the
coordinates were readily available.
191
9
th

The camera location was carefully selected to provide a clear view to show features on the surface and the
underground tunnel being built. The main site constraints in this study were the vegetation obstructing the view
while taking photos, inaccessible location within confines of private residences as well as too high or too far
locations to capture construction site photos in sufficient details.
4.2 On-Site Work
After the locations of the cameras perspective location and the focus point had been evaluated, the photo used for
3D model augmentation was shown in Fig. 4, with the object focus location being indicated at the centre of the auto-
focus point of the camera. The model of the camera was Canon EOS 400D. The camera location and the focus point
were determined. On the site, there were known surveyed control points used to define positions in three dimensions
(Easting, Northing and Zenith). Those points played an important role in planning and design stages, execution of
work and as well as construction monitoring purpose. Well-practiced surveying techniques were used in determining
the coordinates of the cameras location by using surveying instruments, including Total Station, reflector and
measuring tape. The coordinates of the camera and the focus point surveyed were (817537.3, 825641.0, +10.85) and
(817574.5, 825675.1, +3.10) respectively.

FIG. 4: Photo being used in augmentation
4.3 Off-site work
In the virtual reality environment of 3ds Max, 3D models can be built by using the data from as-design drawings.
All the coordinates surveyed are respected in the World Geodetic System. It is difficult to input those coordinates to
the 3ds Max directly. Transforming the global coordinate system to the local coordinate system by using 2D
planar motion transformation of points (Manual of photogrammetry, 5th ed., 143-152) is more convenient.
In the model, the origin is selected at the centre of the cross section of the micro-tunnel for electrical cable. The
transformed coordinate of the cameras perspective location O ( ) and object focus location with
coordinate ( ) are (26.640, 238.713, 15.003) m and (2.560, 194.200, 7.250) m respectively. With the
Normal Vector N, the three orientation angles ( ) have been computed as (-151.6
o
, 81.29
o
, 180.00
o
).
(10)
(11)
With the transformed geometric data, the 3D tunnel model was built within 3ds Max directly. The resulting model
is shown in the Fig. 5. With the precise position and orientation of the virtual camera calculated by the analytical
method, a virtual camera was created and. Fig. 6 shows the position of virtual camera and a virtual photo being
generated in the virtual environment.
Focus point
192
9
th

FIG. 5: As-designed 3D tunneling model FIG. 6: Virtual camera positioning
In the merging process, at least two sets of coordinates in the image coordinate system ( ) in respect of the real
and virtual photos are needed in superimposition by using the "extra point" approach. Thus, by computing at least
one extra points coordinates in the object space, the corresponding coordinates in the image plane could be
evaluated and the grid platform could be generated with the focal point. The actual photo and the virtual photo could
then be merged into an augmented photo by common graphic software, such as Photoshop Free 30-day trail
software (Adobe Systems Inc. 2009).
In this case study, the CCD/CMOS approach has been employed since it was not straightforward to obtain one extra
points position in the real photo. The Canon EOS 400D camera used is a single lens reflex digital camera with a
CMOS APS-C sensor. The dimensions of the sensor are 22.2 x 14.8mm. Thus it was used in setting up the grid in
the image coordinate plane.
By means of inputting one more point from object coordinate system in addition to the focal point, the image of real
and virtual photos could be merged analytically. The underlying computation of rotational matrix Eq. (12) is
introduced in Section 3.
(12)
The point used was the end of the micro-tunnel for electrical cable, the other end of which was the origin as
mentioned before. With the focal length f equal to 18mm, the transformation from the object coordinate (0, 220, 0)
to image coordinate using Collinearity equations are shown in the Eq. (13) and Eq. (14). By plugging in the value (0,
220, 0) to ( ), the image coordinate of the target point was calculated as (8.416180, -6.035576) with
respect to the focal point (0, 0) at the center of the imaging plane. Then merging the two photos was processed on
the grid mapping provided by the photo-editing software Photoshop. The resulting photo is consistent to that
obtained by the manual approach as shown in Fig. 7a.
(13)
(14)
It is recognized that analytical method is an accurate method since the photo-augmenting process is based on
computing with image coordinates. Accuracy can be further improved by calculating more extra target points and
applying least square adjustment technique to adjust the positions and scale of the augmented photo. The major
advantage is that the whole process is computation based, which means automation can be achieved by developing
computer software, without any manual adjustment to superimpose real and virtual photos. The main limitation of
this approach is the image coordinate system must be determined before photo superimposing and augmentation.
The automation cannot be done without the established gird platform.
193
9
th

Further modification was made in highlighting the underground feature of the micro-tunnel by adding a jacking pit
and a receiving pit as shown in Fig. 7b. The jacking pit and receiving pit were constructed by excavation with sheet
pile shoring systems. By adding those pits in the augmented photos, the as-designed underground tunnels could be
clearly seen submerge below the ground level indicated.

FIG. 7a: Augmented Photo FIG. 7b: Augmented Photo with pits
Furthermore, construction site progress can be shown by inserting time dimension to the augmented photos. In
practice, the installed length of the tunnel could be measured by counting the number of pipe segments being jacked.
As-built construction progress could be visualized easily by using construction process records with respect to the
cumulative tunnel length. The total duration of the project was 66 days. The total length of the tunnel was
approximately 220m. Augmented photos representing actual site situation on particular project days are shown in
Fig. 8.

Day 11, length of tunnel is 130m Day 22, length of tunnel is 170m Day 33, length of tunnel is 183m

Day 44, length of tunnel is 198m Day 55, length of tunnel is 203m Day 66, length of tunnel is 220m
FIG. 8: Construction progress visualization
Underground tunnel
Ground level
194
9
th

5. CONCLUSIONS
This paper focuses on the comparison between the analytical technique resulting from recent research and the
manual method for augmenting photos with 3D as-built tunnel models. A real case study was presented to discuss
the site constraints and limitations during site experiments on analytically augmenting site photos with 3D tunnel
models. Then, by computing the cameras position and orientation, the perspective views of the virtual camera and
the real camera have been aligned and the resulting photos superimposed on a regular grid platform. The augmented
photo-based construction site progress visualization was demonstrated also. Contrasting the manual approach, it is
found that the analytical approach in applying photo augmentation is more accurate and potentially programmable
(automation procedure shown in Fig. 9). This new approach lends a high-quality visualization method to benefit
practical applications in construction.

FIG. 9: Conceptual design of automation process
6. ACKNOWLEDGEMENTS
The writers would like to express their sincere thanks to Mr. K.C. Ng, Regional Manager-North, of CLP Power
Hong Kong Limited, Mr. Sam C. K. Shum of The Hong Kong and China Gas Company Limited, Mr. Joseph H. Y.
Kong and Mr. Lotus C.W. Au Yeung of Kum Shing (K.F.) Construction Company Limited, Mr. Eric K.I. Lo and
Mr. William, W.Y. Leung of Black & Veatch Hong Kong Limited, for providing micro-tunneling construction site
access and first-hand data. Thanks are also due to Mr. F. Dai for his guidance, and thought-provoking comments.
This presented research was substantially funded by Hong Kong Research Grants Council (RGC) through Project
PolyU 5245/08E.
7. REFERENCE
Adobe Systems Inc. (2009). "Featured product downloads. " Available via < http://www.adobe.com/downloads/>
[access: 01-Jun-09]
Autodesk Inc. (2009). "3ds Max: Production-proven 3D modeling, animation, and rendering solution for games,
film, television, and digital publishing." Available via <http://usa.autodesk.com/adsk/servlet/index?id=
5659302&siteID=123112> [access: 01-Jun-09]
Behzadan A. H. and Kamat V. R. (2007). "Georeferenced Registration of Construction Graphics in Mobile Outdoor
Augmented Reality. " Journal of computing in civil engineering, 21(4), 247-258.
Dai F. and Lu M. (2009). "Analytical approach to augmenting site photos with 3D as-built bored pile models. " In
proceeding of the 2009 Winter Simulation Conference (M. D. Rossetti, R. R. Hill, B. Johansoon, A. Dunkin
and R. G. Ingalls, editors).
Output Computer Automation Process Data Input
Coordinate of cameras
perspective location
O ( )
Coordinate of focus point
( )

Extra point in virtual photo
+ Extra point in real photo
or CCD/CMOS size
Calculation of
orientation angles

Setting up grid
platform
Superimposition
of virtual and real
photos
Augmented Photo
195
9
th

DALSA Corporation. (2009). "CCD vs. CMOS. " <http://www.dalsa.com/corp/markets/CCD_vs_CMOS.aspx>
[access: 28-Jun-09].
Kensek K, Noble D., Schiler M. and Tripathi A. (2000). "Augmented Reality: An Application for Architecture. "
Proceedings of the Eighth International Conference on Computing in Civil and Building Engineering, 294-
301.
Kim H. and Kano N. (2008). "Comparison of construction photograph and VR image in construction progress. "
Automation in Construction, Vol. 17, No. 2, 137-143.
Matossian M. (2004). "3ds max 6 for Windows. " Berkeley, Calif.:, PeachPit Press. 375-459.
McGlone J. C., Mikhail E. M., Bethel J. and Mullen R. (2004). "Manual of photogrammetry. " 5th ed. Bethesda,
Md.: American Society for Photogrammetry and Remote Sensing. 1-2, 143-152
Murdock K. (2008) ."3ds Max 2008 bible. " Indianapolis, Ind.: Wiley Pub., Inc.. 35-324
Sin D. H. and Dunston P. S. (2008). "Identification of application areas for Augmented Reality in industrial
construction based on technology suitability. " Automation in Construction, Vol. 17, No. 7, 882-894
Wang X. Y. and Dunston P. S. (2005). "Real Time Polygonal Data Integration of CAD/Augmented Reality in
Architectural Design Visualization." Proceedings of the 2005 ASCE International Conference on Computing
in Civil Engineering, 1-8.

196
9
th

AUTOMATIC GENERATION OF TIME LOCATION PLAN IN ROAD

CONSTRUCTION PROJECTS

Raj Kapur, Shah, PhD Candidate:
CCIR, School of Science and Technology, University of Teesside, Middlesbrough, TS1 3BA, UK,
Email: r.k.shah@tees.ac.uk

Nashwan Dawood, Professor:
CCIR, School of Science and Technology, University of Teesside, Middlesbrough, TS1 3BA, UK,
Email: n.n.dawood@tees.ac.uk
ABSTRACT: Because of unique characteristics of earthwork activities in road projects, construction managers
and planners require innovative techniques to assist them in producing accurate planning tasks such as efficient
resource allocation and costing of activities. The research study introduced a framework of an innovative model
dubbed Virtual Construction Model (VCM). The paper focuses on automatic generation of a time location
plan and conflict identification system for earthwork activities in a road project. The framework is designed by
integrating road design data, quantities of cutting and filling sections, variable productivity data, algorithms for
4D terrain modelling, and a time location plan generator. The model is validated with a real life case study of a
road project and it was found to be beneficial in generating the 4D terrain surfaces of progress and a time
location plan with more accurate information of location and quantities in the earthwork operations. The VCM
has potential to assist project planners and construction managers in producing efficient construction
scheduling and resource planning.

KEYWORDS: Earthwork, Productivity, Sensitivity Analysis, Time Location Plan, Virtual Construction Model
1. INTRODUCTION
The planning, scheduling, and controlling system adopted by project planners and construction managers
determines the success of any construction projects. Construction managers and project planners of linear
construction projects such as roads, railways, and pipelines require advanced project planning and scheduling
tools to control budget, schedule and resource allocation so that project goals could be achieved on time and on
budget. The effective application of planning and scheduling techniques such as CPM and PERT is limited
because road construction activities are fundamentally different to building construction projects (Hamerlink
and Yamin, 2000).

In a large-scale project, a visual representation of the construction schedule can be extended to monitoring not
only the construction progress, but also all the auxiliary activities, including onsite plant and equipment (Adjei-
Kumi et al, 1996). McKinney et al (1998) demonstrated the capability of 4D-CAD models to identify the
construction problems prior to their actual occurrence. The failure to provide the information of spatial aspects
of a construction project by traditional techniques such as Bar Charts and the Critical Path Method (CPM) have
motivated the research effort to incorporate visualisation techniques into project scheduling and progress
control (Koo and Fischer, 2000). Zhang et al (2000) further developed a 3D visualization model with schedule
data at the level of construction components. Dawood et al (2002) developed an integrated database to act as an
information resource base for 4D/VR construction process simulation and it was applied to a building project.

Furthermore, several research efforts carried out in the visualisation of the construction process applied to
building construction projects, but there have been limited research studies in the area of infrastructure
construction projects. For example, Liapi (2003) focused on the use of visualisation during construction of
highway projects to facilitate collaborative decision making on construction scheduling and traffic planning,
however, the visualisation of the construction schedule for the intermediate stages of the construction process
was neglected. Castro and Dawood (2005) developed the RoadSim simulator based on the site knowledge-
based simulation system. It is applicable to develop a master construction schedule in a road project based on
simulated productivity of road building activities and available resources with different sets of equipment and
site working conditions. Kang et al (2008) suggested an approach to simulate 4D models for the movement of
197
9
th

earthwork activity for the intermediate stage of the construction process in civil engineering projects using
morphing techniques and realisation of construction progress in graphical images. The 4D models of earthwork
operation have been produced in 3D CAD model at equal volume and at a fixed production rate of the
earthwork activities at different stages during construction operation and linked with time but the variable
productivity data of equipment and soil characteristics was not considered in the 3D CAD models.

The above research efforts did not address the interface of variable production rate, which depends on available
resources and site conditions for the development of the VCM. The key issue faced in road construction sites is
the variable productivity from one day to another due to the special characteristics of the road construction
industry; such as fluctuation in daily weather conditions, working conditions in open sky, resource
unavailability on time and other unpredictable factors. The study focuses on addressing the above issues by the
development of the VCM. The model will be integrated with the variable production rate of earthwork
activity throughout road construction operations. Currently accepted scheduling techniques including CPM,
PERT and Bar Charts are unable to model linear activity more accurately in terms of locations. A linear
scheduling method developed earlier than CPM has the potential to provide significant enhancement, because it
provides location of working activities coupled with the advancement of computer technology. This allows the
project schedulers and construction managers to plan road construction project visually to determine the
controlling activity path (Hamerlink and Yamin, 2000).

Previously research efforts by Johnston (1981 and Garold et al, 2005) in the area of linear scheduling concluded
that the techniques are a useful scheduling tool for progress monitoring in road construction projects during the
planning and execution phases. Previous research studies have considered earthwork activities as a linear
activity (Hamerlink and Yamin, 2000). However, earthwork activities are nonlinear in real practice since the
earthwork quantities vary along a road project from station to station (chainage to chainage) according to
topography of terrain surfaces. To overcome this issue of the earthwork activity, this study presents an
innovative methodology for the development of VCM and a time location plan of the earthwork construction
processes in road construction projects. The model intends to assist in improving the site communications of
road construction planning and scheduling information, and to produce efficient construction scheduling and
resource planning. The remainder of this paper outlines a conceptual framework and details of the prototype for
VCM and generation of a time location plan of the earthwork operations in a road project.

2. FRAMEWORK OF A VIRTUAL CONSTRUCTION MODEL (VCM)
The general specification of framework of a prototype of virtual construction model is outlined in figure 1. The
framework integrates the road design data, sectional quantities of cut and fill, productivity models, algorithms
for 4D terrain modelling and a time location plan generator. The model assists in generating visual terrain
surfaces of road progress automatically throughout the earthwork operations. The next section describes in
detail the input, process and output of the VCM.
FIG. 1: Framework specification of a Virtual Construction Model (VCM)
198
9
th

2.1 Input

The sectional quantities of cut/fill of earthwork activity along a road section, productivity of the activities and
construction site knowledge base are key components of the framework. The sectional cut/fill quantities are
calculated using road design data including L-sections and X-sections at required intervals of chainage. The
productivity data, which is produced by the RoadSim simulator, is used as a key input in the model. The
productivity is calculated using the available resources, equipment sets and site working conditions. This is
incorporated with the model to determine the total duration of the earthwork operations. The soil characteristics
along the working road section, types of available equipment set for a selected activity, haulage distance of soil,
working conditions and all other factors including weather conditions that control productivity has already been
incorporated within the RoadSim simulator. Additionally, the model will assist in identifying the possible
location and numbers of site access points. The construction knowledge will assist to select the methods of
construction process for different types of soils, equipment sets for a particular activity and soil characteristics.
The site operational rules allow in establishing the sequential relationships amongst listed activities during the
construction operations. The following section describes and demonstrates the process of the VCM.
2.2 Process

The process of the framework includes four modules: data generation module, visualisation module, cost profile
module and a time location module. Data generation module processes the input data to produce a detailed
schedule and to generate the coordinate data based on the production rate i.e. on the weekly or daily basis in
this study. The visualisation module processes the coordinate data produced by the data generation module, and
converts it into terrain surfaces of the road progress profiles. The cost profile module generates weekly cost
profiles/histograms and the time location module generates a time location plan of the earthwork operations in
road projects. This paper focuses on the development of time location module and conflict identification system
are presented in the following sections.

2.2.1 Generation of Time Location Plan
In this section, a set of algorithms is developed to automate the generation of terrains of earthwork activities at
different stages of the construction process. This is considered the quadratic equations, which determines the
progress height for earthwork activities. A detailed development for the generation of weekly coordinate data
presented by Shah et al (2008) is used for the generation of a time location plan in this research study.

The time location plan is also known as time-distance planning, time-chainage planning and linear scheduling
method (LSM). It enables the creation and display of planning and scheduling information of earthwork
activities in two dimensions: Location in X-axis and Time in Y-axis or vice versa together with the
topographical information of a road project. The slope of activities displayed in time location plan represents
the rate of productivity. The slope of activity provides the early indication of conflicting or overlapping
activities that may occur during the course of activity progress.

An algorithm was designed to identify the start and end location as well as start and end time of earthwork
activities. The developed algorithm determines the location (chainage) along a road section and are broken
down into weekly schedules satisfying the linearity characteristics (start and end locations having equal
production rate) of the earthwork activities. The identified locations and time are summarised and presented in
a table. A linear schedule of the earthwork activities is generated from the tabulated locations and time as
(coordinates of the starting and ending points of weekly earthwork activities) developing a module (macro)
based on using Excel VBA (Visual Basic for Applications) programming.

The generated time location plan provides clearer representation of a construction schedule and enables the
visualisation and analysis of the status of construction activities on a particular location along the road sections.
It also supports in identifying the possible conflicting/overlapping locations along the road section. The detail
of the development of conflict identification system is described in the following sections.

199
9
th

2.2.2 Development Conflict Identification System (CIS):

This section focuses on exploring and developing a new methodology in which the VCM can be enhanced to
represent earthwork scheduling and planning information in a time location plan (TLP) and to identify the
possible conflicting points. It is envisaged that TLP will provide more accurate information in terms of location
through integrating with a real site operations and incorporating actual earthwork progress data with the VCM.
In this way, the model enables the integration of real site data of soil profiles and assists to update site
productivity in earthwork operations according to soil characteristics along a road project and scheduling
information is represented in a time location plan. The flow diagram of the conflict identification system (CIS)
is presented in figure 2 and the details of the flow diagram are described below.

The conflict identification system is important to construction managers and project planners in order to
identify the possible overlapping activities in advance so that space conflicts, wastage of resources, idle
equipment and reduction in site productivity can be resolved at planning stage. It is anticipated that conflict
identification system can assist to reduce the remaining difficulties encountered by construction managers when
allocating resources and monitoring the site progress. To resolve the above issue, this research study focuses on
a conflict identification system in a road project including earthwork operations. The functionality of VCM has
been improved by incorporating a new approach, which is useful to identify and determine the overlapping or
conflicting points in terms of location and time along a road section between two activities in earthwork
operations.
FIG. 2: Data flow diagram of Conflicting Identification System (CIS)
200
9
th

Location (L)
Time (L)
Slope (Productivity) of earthwork Activity A1 = m1
Slope (productivity) of sub base activity A2 = m2
A (L1, T1)
B (L2, T2)
C (L3, T3)
D (L4, T4)
Conflict point (L, T)
The conflicting identification system (CIS) is expected to assist project planners and construction managers in
identifying and determining the coordinates (location and time) of the conflicting/overlapping points having
different productivity value and passing through different site access points. The location and time of
conflicting/overlapping activities during earthwork operations can also be analysed considering the soil
characteristics along a road section.The system enables project planners and construction managers to take
preventive measures by forecasting the possible conflicting activities in earthwork operations. If planned
productivity varies time progression due to variation in soil characteristics or site access points along the road
projects, there is a chance of overlapping or confliction between activities with different productivity. The
following section describes the detailed derivation of mathematical formula for the identification of conflicts
between two earthwork activities.

2.2.3 Derivation of mathematical formula for conflicting location and time:

FIG. 3: Time Location Plan for earthwork activities A1 and A2

Considering figure 3 for determining the conflicting/overlapping point between activity A1 and A2:
Assume, Line AB represents earthwork activity (A1) which is passing through point A (L1, T1) and B (L2, T2),
and Slope of the line AB is represented by m1 . Similarly, Line CD represents sub base activity (A2) which is
passing through point C (L3, T3) and D (L4, T4), and Slope of the line CD is represented by m2

Slope of line AB (activity A1) = m1 = (T2-T1) / (L2-L1) .. (1)
i.e.; Productivity of earthwork activity A1 is expressed in linear metre/week
Similarly, Slope of line CD (activity A2) = m2 = (T4-T3) / (L4-L3) ... .. (2)
i.e.; Productivity of earthwork activity A2 is expressed in linear metre/week whereas; C1 and C2 are the
intercept of line AB (activity A1) and CD (activity A2)

According to coordinate geometry, Eq. of a straight line passing through point (x, y) and having slope (m) and
intercept c is y = m x + c. Similarly, equation of a straight line AB having
Slope (m1), intercept C1 and passing through point (L1, T1) is expressed in equation 1 below
T1 = m1*L1+C1 . (3)
Equation 3 is derived by algebraic transformation of equation 1;
C1 = T1- m1*L1 (4)
If the line AB is passing through a point (L, T), the equation of the line AB as below:
T = m1* L + C1 . (5)
Substituting the value of C1 in Eq. 3;
T = m1 *L + T1 - m1*L1;
Or, T = m1 *(L - L1) + T1 ... (6)
201
9
th

Similarly, Equation 7 can obtain after deriving, equation of line CD having slope (m2) and passing through (L3,
T4) is; T = m2 *(L-L3) + T3 (7)
Equations 8 and 9 can be obtained by solving above equations 4 and 7 for identification of
conflicting/overlapping location at distance (L) and Time (T)
L = [(m1*L1-m2*L3) + (T3-T1)]/ (m1-m2) (8)
T = [m1* m2 *(L1-L3) + m1*T3 m2*T1]/ (m1-m2) (9)

Therefore, equation 8 will provide the location where two activities overlap each other and equation 9 will
provide a time when both activities overlap each other at the conflicting location. An illustration of the system is
presented in the following section.
2.2.4 Demonstration of conflict points calculation:

FIG. 4: Two typical work activity having different productivity

Assume, line AB represents earthwork activity (A1) of earthwork at week 1 which is passing through starting
point A (100 m, 1wk) and ending at point B (200 m, 2 wk). similarly, line CD represents sub base activity (A2)
of earthwork at week 2 which is passing through starting point C (75 m, 2wk) and ending at point D (225 m, 3
wk) as shown in figure 4. According to the slope of a straight line; m = y2-y1/x2-x1. Therefore, Slope of the line
AB (activity A1) = m1 = (T2-T1)/ (L2-L1) = (2-1) / (200-100) = 0.01 m/wk,
Similarly, Slope of the line CD (activity A2) = m2 = (T4-T3)/ (L4-L3) = (3-2) / (225-75) = 0.0067 m/wk

Using equation 8, the conflicting point at the location (L) is determined as below;
L = [(m1*L1 - m2*L3) + (T1-T3)]/ (m1-m2) = [(0.01*100 - 0.00667*75) + (1-2)] / (0.01-0.00667) = 450.04 m.
Furthermore, the conflicting/overlapping at time (T) is determined by equation 9. T = [m1* m2 *(L1-L3) +
m1*T3 m2*T1]/ (m1-m2) = [0.01*.0667(100-75) + 0.01*2- .00667*1]/(0.01-.00667) = 4.5 wks. Therefore, the
activities A1 and A2 will conflict or overlap at point (L, T) = (450 m, 4.5 wks)

3. DEMONSTRATION OF THE VCM WITH A CASE STUDY

3.1 Case study development:
A real life case study involving 1.5 km of road section of lot no. 3 road project in Portugal was selected and
demonstrated the model for earthwork activity of cut to fill or spoil. For this purpose, actual road design
parameters and geometric data of the L - section and the X-section is considered, and the sectional quantity of
earthwork is calculated assuming the typical trapezoidal sections at 25 m intervals along the selected length of
road section. The maximum point of cut/fill section is identified where construction operations start first as per
existing practice and construction site knowledge. Innovative arithmetic algorithms and derived mathematical
equation for the height calculation is designed, developed and validated by the authors The detailed derivation
of the equation and algorithms was discussed and presented in Shah et al (2008).

In this case, progress height is presented by Z- coordinate whereas X direction represents along the road and Y
direction represents along the cross section. The road surface is presented in terms of height in mesh form. The
productivity of the selected activity is the key variable to identify the next surfaces/layers in the construction
progress. The next surface/road profile has been developed based on remaining sectional quantity after progress
of earthwork equivalent to the weekly production rate. The operations repeats for the next economical stretch of
length where the cutting and filling operations take place in order to generate earthwork progress profiles
A=100,1
Conflictingpoint
D=225,3
C=75,2
B=200,2
202
9
th

automatically of a road section. The economical stretch (balance line) has been identified using the mass haul
diagram rules and it is used in the model.
3.2 Time Location Plan Generated by the VCM:

The developed VCM has additional capability to generate a time location plan as a construction planning and
scheduling tool for a road section throughout the earthwork operations as shown in figure 8. The coordinates of
starting and ending location of earthwork activities with corresponding start and end date in week or day is
generated as shown in table 1 using an algorithm and integrating it with the VCM. The data of the table is used
to generate a time location plan as shown in the figures 9 (a) and (b) using VBA programming. The time
location plan is integrated with variable productivity data so that the time location plan can be generated
automatically according to any changes in site conditions, access point, equipment productivity and soil
characteristics throughout the construction operations.
Table 1: coordinate data of start, end location, start, and end date for cut/fill activities
S.N. X1 (Start
Station) m
X2 (End
Station) m
Y1 (Start
Date) day
Y2 (End
Date) day
Cut/Fill
w0 125 200 0 1 F
w1 100 200 1 2 F
w2 100 200 2 3 F
w3 100 200 3 4 F
w4 0 200 4 5 F
w5 0 225 5 6 F
w6 0 225 6 7 F
w7 0 225 7 8 F
w8 0 225 8 9 F
w9 0 225 9 10 F
w10 0 225 10 11 F
w11 0 225 11 12 F
w12 0 225 12 13 F
w0 325 425 0 1 C
w1 325 475 1 2 C
w2 300 475 2 3 C
w3 300 475 3 4 C
w4 300 475 4 5 C
w5 275 475 5 6 C
The number of weeks required for a cutting or filling section is represented by weeks such as w1 (filling at day 1
in blue colour), w1 (cutting at day 1in green colour) as shown in index of figures 5 (a) and (b). Figure 5(a)
shows when using two set of filling and one set of cutting equipment and figure 5(b) shows when using one set
of filling and one set of cutting equipment. Similarly, the coordinate of starting and ending point of activity is
also presented in terms of location and time (m, day) as shown in figures 5 (a) and (b) including with
comparisons with existing time location shown on dotted line.
FIG. 5 (a): Snap shot of automatic generated time location plan using two sets of filling and one set of cutting
equipment for earthwork operations.
Filling activity
Cutting activity
Existing time location plan of cut/fill activity
203
9
th

The conflicting identification system (CIS) is expected to assist project planners and construction managers in
identifying and determining the coordinates (location and time) of the conflicting/overlapping points having
different productivity value and passing through different site access points. The location and time of
conflicting/overlapping activities during earthwork operations can also be analysed considering the soil
characteristics along a road section. The system enables project planners and construction managers to take
preventive measures by forecasting the possible conflicting activities in earthwork operations. If planned
productivity varies time progression due to variation in soil characteristics or site access points along the road
projects, there is a chance of overlapping or confliction between activities with different productivity. The
following section describes the detailed derivation of mathematical formula for the identification of conflicts
between two earthwork activities.
FIG. 5 (b): Snap shot of automatic generated time location plan using one set of filling and one set of cutting
equipment for earthwork operations.

3.3 Sensitivity analysis of earthwork duration
The sensitivity analysis of earthwork duration presented below demonstrate that the capability of the model to
assist in analysing with what if scenarios for different variable factors including site access points, type of
equipment and soil characteristics along a road section that affect the development of a time location plan and
resource planning in the earthwork operations.

3.3.1 Sensitivity analysis of earthwork duration due to site access points:
The result presented in figure 6 shows that total duration was 7 days for filling-1 & 2 sections and 3 days for
cutting-1 section of earthwork operations, which is similar for access points 3 and 5 assuming that other
variables and resources are constant. However, the total duration was 9 days for filling-1 & 2 sections and 4
days for cutting-1 section of earthwork operations for six number of access points, which are higher than the
smaller number of, access points. Therefore, the conclusion from these results is that lower number of access
point is more economical, less time and resources consumption to complete the same quantity of earthwork in
comparison to a higher number of access points for same section in a road construction project.

FIG. 6: Sensitivity analysis result of total duration for
variable of site access points
FIG. 7: Sensitivity analysis result of total duration
for variable of equipment types
Existing time location plan of cut/fillactivity
Filling activity
Cutting activity
204
9
th

3.3.2 Sensitivity analysis of earthwork duration due to equipment types
The results presented in figure 7 above show that total duration was 35 days for filling 1 & 2 sections and 18
days for cutting -1 section of earthwork operations at excavator type -1(Exa). Whereas, total duration was 35
days for filling 1 & 2 sections and 14 days for cutting-1 section of earthwork operations at excavator type -2
(Exb) assuming other variables and resources are constant. However, the total duration was 34 days for filling-1
& 2 sections and 12 days for cutting -1 section of earthwork operations when using excavator type -3 (Exc),
and total duration was 26 days for filling -1 & 2 sections and 9 days for cutting-1 section of earthwork
operations using excavator type-4 (Exd). Therefore, the above results revealed that higher productive equipment
is more economical and need less time and resources to complete the same quantity of earthwork in comparison
to lower productive equipment in same section of a road project and under similar circumstances. Since the
mobilisation and demobilisation cost is same for each type of equipment, it will be more economical and logical
to plan and use higher productive equipment for the earthwork planning if site conditions allow the operation of
higher productive equipment.

3.3.3 Sensitivity analysis of earthwork duration due to site soil characteristics
The result presented in figure 8 shows that total duration was 40 days for filling-1 & 2 sections of earthwork
operations since the soil characteristics for all filling sections were same throughout the road section. However,
the total duration for cutting section for different types of soil was different. The result shows that duration of
cutting-1 sections are 20, 19, 21 and 23 for different types of soil for sand, sand-clay, clay dry and clay wet
respectively. Therefore, the above results confirm that sand-clay soil at cutting section requires less time in
comparison with sand, clay-dry and clay-wet whereas soil types such as clay-wet need more time to complete
the same quantity of earthwork under similar site constraints in comparison to other soil characteristics.

FIG. 8: Sensitivity analysis result of total duration for variable of soil types
From above result, it is anticipated that the sensitivity analysis will assist project planners and construction
managers in simulation analysis with what-if scenarios for the production of efficient construction scheduling
and resource planning under different site conditions including types of equipment, soil characteristics and site
access points in a road construction project. The results of sensitivity analysis confirm that all three variables
(site access points, types of equipment and soil characteristics) are the most critical variables, which have direct
impact in productivity and resource planning for earthwork operations in a road project. Therefore, project
planners and construction managers need to analysis and simulate more carefully at detailed planning stage for
the production of efficient construction scheduling and resource planning in the earthwork operations in a road
construction project.

4. CONCLUSIONS
The paper has introduced an innovative methodology for the development of a prototype model dubbed as
Virtual Construction Model (VCM) aiming to produce and visualise earthwork progress profiles and a time
location plan throughout the earthwork operations in road construction projects. The developed model has
capability to generate progress profiles according to variable productivity data and at a particular time
considering it 4
th
dimensions, which is derived from the productivity of earthwork activities. This is considered
205
9
th

as an innovative approach for the VCM comparison to 4DCAD technology where variation of earthwork
productivity due to site conditions and soil characteristics was not integrated with the 4DCAD models. The
model also generates visual representation of construction progress and a time location plan of construction
project showing an accurate location of the earthwork activities and corresponding time. The model has
capability in interfacing with user-defined variables including soil characteristics and site access points
according to topographical constraints, which is considered as another key achievement of this research study.
The sensitivity analysis presented in the paper with a case study confirms that the model assists project planners
and construction managers to analyse what-if scenarios with soil characteristics and resource constraint
through the visual simulation in construction scheduling and resource planning processes.

The paper concludes that the VCM introduced by the research is a decision support tool for earthwork
construction management. The model will facilitate a logical decision-making process for the earthwork
construction scheduling and resources planning tasks in improving site productivity and reducing the production
cost of earthwork operations in road projects.

5. ACKNOWLEDGEMENTS
This research is supported by the Centre for Construction Innovation and Research (CCIR), University of
Teesside, UK and Portugal based International Construction Company, MOTA-ENGIL. Sincere appreciation is
given to the sponsor from the authors.

6. REFERENCES
Adjei-Kumi, T., Retik, A. and Shapira, A. (1996), Integrating on site tasks into planning and scheduling of
construction projects., Proc., shaping theory and practice, managing the construction project and managing
risk: Int. Symp. on organisation and management of construction, D.A. Langford and A. Retik., eds., E &
FN Spon, Glasgow, U.K., 283-292.

Castro, S. and Dawood, N. (2005), RoadSim: A Road Construction Knowledge-Based Simulation System,
Proceedings of the CIB W102 Conference, 2005, Lisbon, Portugal.

Dawood N., Eknarine S., Mallasi, Z. and Hobbs, B. (2002) Development of an integrated information resource
base for 4D/VR construction processes simulation Automation in Construction, (12) pp 123-131.
Garold, D.O., Samir, A. A. and Gregory, A. D. (2005), Linear scheduling of highway project, Oklahoma
Transportation Centre Research Project, OTC-25.

Harmelink, D. J. and Yamin, R. A. (2000), Development and application of linear scheduling techniques to
highways construction projects, Technical report of FHWA/IN/JTRP-2000/21, Oct. 2000, pp8-69.

Johnston, D. W. (1981), Linear Scheduling Method for Highway Construction, Journal of Construction
Division, ASCE, 107 (2) pp. 247-261.

Kang, L. S., Moon, H. S., Moon, S. J. and Kim, C. H. (2008), 4D System for visualisation scheduling progress
of Horizontal Construction project Including Earth work, Proceedings of the CONVAR Conference 2008,
21-22 October, Kula Lumpur, Malaysia.
Koo, B. and Fischer, M. (2000), Feasibility study of 4D CAD in commercial construction, Journal of
Construction Engineering and Management, 126 (4), 251-260.

Liapi, A. K. (2003), 4D visualisation of highway construction projects Proceedings of 7
th
International
Conference on Information Visualisation (IV03), 1093-9547/03 2003 IEEE.

McKinney, K. and Fischer, M. (1998), Generating, evaluating, and visualizing construction schedules with
CAD tools, Automation in Construction, 7(6), 433447.

Shah, R. K. , Dawood, N. and Castro, S. (2008), Automatic Generation of Progress Profiles for Earthwork
Operations using 4D Visualisation Model, Electronic Journal of Information Technology in Construction,
IT Con. Vol. (13), PP 491-506.
Zhang, J. P., Anson, M. and Wang, Q. (2000), A new 4D management approach to construction planning and
site space utilization. Proc., 8th Int. Conf. on Computing in Civil and Building Engineering, ASCE,
Reston, Va., 1522.
206
9
th

DEVELOPMENT OF 3D-SIMULATION BASED GENETIC
ALGORITHMS TO SOLVE COMBINATORIAL CREW ALLOCATION
PROBLEMS
Ammar Al-Bazi, PhD Candidate,
School of Science and Technology, University of Teesside;
a.f.albazi@tees.ac.uk, http://tees.academia.edu/AmmarAlBazi
Nashwan Dawood, Professor,
n.n.Dawood@tees.ac.uk, http://tees.academia.edu/NashwanDawood/
John Dean, Project Officer,
J.Dean@tees.ac.uk
ABSTRACT: This paper presents an innovative approach to solve combinatorial crew allocation problems in any
labour-intensive industry. This possibly can be achieved by combining 3D-simulation technology with Genetic
Algorithm (GA). GA is one of the Artificial Intelligent tools that were successfully used in optimising performance of
simulation models. The integrated system can determine the least costly and most productive crews to be assigned to
production processes. Discrete Event Simulation (DES) methodology is used to develop a 3D-simulation model that
conveys the idea of how a labour-driven manufacturing system works. A proposed GA-based Multi Layers
chromosome is developed to be integrated within the developed simulation model. This type of integration can
optimise performance of the developed simulation model, through guiding it toward better solutions. The concept of
Multi-Layers chromosome is proposed in order to store different sets of labour inputs such as (daytime shift crew,
night shift crew, process priority, etc). GA operators are developed to ensure more random search for promising
solutions in a large solution space. As a case study, a sleeper precast concrete manufacturing system is chosen to
prove the concept of the proposed system. The results showed that adopting different allocation plans had a
substantial impact on reducing total allocation cost, process-waiting time, and optimising resource utilisation. In
addition, worker utilisation and process-waiting time have a significant effect on the labour allocation cost.
KEYWORDS: 3D-simulation, Multi-Layers Genetic Algorithms, Crew Allocation Problem, Precast Concrete
Manufacturing System

1. INTRODUCTION
Crew allocation is the process of deciding where and when, crews can be assigned according to their required
qualification or skills. Crew consists of a composition of different skilled workers to carry out a certain process in a
production facility. This term is used in processes, which needed intensively human resources to carry out jobs.
The complexity of crew allocation problem appears when different production processes demand the same type of
resource (worker or machine) at the same time. Such competition on using resources has the potential to cause
process-waiting time, labour idle time, low resource utilisation, and subsequently high labour allocation cost. Such
allocation problems can be seen in a job-shop environment such as seen in the precast concrete industry.
Many analytical assignment models have been developed to solve allocation problems during the last 50 years,
Pentico (2007). Many real life allocation problems are complex and involve a huge number of alternatives to be
modelled, this alternative explosion causes a combinatorial problem. To avoid the problem of combinatorial
explosion, many heuristic rules and artificial intelligent tools have been developed to solve such problem.
Crew allocation system seems to be a very visible and economically significant tool to solve such complex
allocation problem. The crew allocation system can be used as a decision support system in crew planning and
207
9
th

scheduling of labours in many manufacturing systems where human resource is a substantial resource. The use of
such an allocation system can assist production managers to identify the best allocation plan in a short time.
In this paper, an innovative crew allocation system dubbed SIM_Crew is developed. Process Simulation and
Artificial Intelligent technologies have been integrated to produce a sophisticated crew allocation system. GA is
suggested as a promising optimisation engine to be embedded within simulation model for better searching. An
intelligent searching algorithm is developed to explore a large solution space. Multi-Layers chromosome that can be
used to store and manipulate different sets of various attributed data is proposed. A special selection rule is designed
to give a higher selection chance for promising chromosomes for further investigation. Dynamic Probabilistic
Crossover (DPC) is developed to enable stochastically the exchanging of genes at different chromosome layers. In
order to avoid the local minimum trap, avoid chromosome repetition, and to add more randomness to the searching
process, a Dynamic Probabilistic Mutation (DPM) is developed to exchange randomly the current proposed crew for
a process with any of the crew alternatives for that process.
This paper is organised as follows: In section 2, the literature review exploring crew allocation techniques is
presented. In section 3, the specification of the crew allocation system is demonstrated. In section 4, a real life case
study is presented. Conclusion and future development are the contents of the last section.

2. RELATED WORK ON CREW ALLOCATION PROBLEMS
Crew planning and scheduling using simulation modelling and optimization technology has been charted in the
works of Lu, et. al (2005) who presented a computer system called simplified simulation-based scheduling (S3) to
solve the problem of skilled labourer scheduling in a multi-project context. Marsono (2006) developed a simulation
model for the production of Industrialized Building System (IBS) components. Dawood, et. al (2007) developed a
generic simulation model depicting the operational processes of precast concrete production systems. The simulation
model was developed to study cost and a trade-off schedule under different resource allocation policies, resource
utilisation evaluation was considered in this study. Nassar (2005) presented a model that uses spreadsheet GA
implementation to optimally assign resources to the repetitive activities of construction projects, in order to
minimise the overall project duration as well as the number of interruption days. Ipsilandis (2006) presented a linear
programming parametric model formulation for supporting the decisions of construction managers: explored the
multi-objective nature of decision-making in repetitive construction projects. Watkins, et al. (2007) used agent based
modelling methods to simulate space congestion on a construction site to explore the impacts of individual
interactions on productivity and labour flow. In this simulation, two masonry crews intersecting in space are
considered. Li, et. al (1998) presented a methodology for optimising labour and equipment assignment for
excavation and earthwork tasks using a Genetic Algorithm. Moselhi et. al (2007) proposed a new methodology
which uses combined Genetic Algorithms and spatial technologies for optimisation of crew formations for
earthmoving operations. Marzouk, et. al (2007) presented a special purpose simulation model to capture the
uncertainty associated with bridge construction. A sensitivity analysis was performed to study the impact of
assigned labour crews in the estimated durations of segment fabrication and deck construction. The studies above
presented a number of methodologies and tools that have been developed so far to manage and schedule crews in a
number of labour-intensive industries.
More systems that are sophisticated seem to be needed to advance the practice of crew planning and scheduling in
any labour-intensive industry. As more analyses are required to identify the behaviour of allocation solutions,
analysis of effects of resource utilisation and other performance indicators on reducing labour allocation cost are
required to be enabled in such sophisticated systems. In the next section, the specification of the proposed allocation
system and SIM_Crew modules will be described in detail.

3. THE CONCEPTUAL MODEL OF THE PROPOSED ALLOCATION SYSTEM
Conceptual model can be defined as a visual diagram of representing a set of causal relationships among proposed
components that are believed to form a desired integrated system for solving a particular problem.
The purpose of developing a conceptual model is to capture all possible causal relationships among the real-world
entities, which are concerned with the core functionality of the system being proposed. The process of constructing a
208
9
th

conceptual model starts with an analysis of the entity requirements of the real world, to identify those entities which
are of key significance to the system, excluding any that are irrelevant or unrelated to its core functionality.
In this study, a conceptual model is proposed to introduce an innovative crew allocation system, this model shows
how all components of the integrated system can work together in order to process inputs and produce outputs. The
purpose of manipulating crew allocation is considered as a trial to place the right and most suitable crew to a process
so minimum allocation cost is achieved. Therefore, as a fundamental requirement, crews should be allocated to
production processes with minimum delays that might caused by shared labourers of different crews to carry out
same job at the same time. In addition, the performance of alternative feasible allocation plans should be evaluated
in terms of labour allocation-related cost, so that the efficient one with the minimum labour allocation cost and
reduced process-waiting time is selected, for implementation as the best crew allocation plan. To satisfy the above
allocation process objective, system specification is developed which involves integration of simulation technology
and Genetic Algorithms as a processing core shown in figure 1.

FIG. 1: SIM_Crew Conceptual Model
In figure 1, SIM_Crew consists of a simulation model, which is integrated with databases through the integration
and processing module (this module is developed by writing a VBA codes). Two types of databases are developed to
provide the simulation model with the required information about product specifications and other related labour
information. An optimization module is designed to be embedded in the simulation model: the function of this
module is to provide simulation with high quality feasible allocation plans for evaluation purpose.
Many key performance indicators such as labour allocation costs, process-waiting time, and utilisation of labourers
are considered as outputs. The interface mechanism is designed to include size of population, selection strategy,
optimisation engine operators (crossover and mutation), and other requirements. As shown in figure 1, the allocation
process is an iterative procedure of progressive improvement in which GA module proposes more than one
allocation plan to be evaluated by the simulation model, results are fed back to the optimisation engine to decide and
propose according to the efficiency more promising allocation plans. During allocation iterations, simulation
executes allocation plans: each of them consists of a set of proposed crews to be allocated to production processes,
while GA evaluates the performance of the resultant allocation, and based on this, adjusts the decision variables and
selects the most promising. A conceptual flexible model for development of intelligent simulation in manufacturing
systems was presented by Azadeh, et. al (2006). The SIM_Crew modules will be described in the following sub-
sections:
3.1 Constructing a Relational Database Model
In order to ensure a quick locate or access to any specific crews member, crew alternatives and other worker
specifications, a relational database model is developed for such purpose. In addition, the purpose of developing a
relational database model is to structure the storage of information in a way that future development such as adding
209
9
th

or removing a process, worker, or any other related detail would be easier and flexible. In addition, Structured Query
Language (SQL) facility, which Access database can provide was useful to ease the searching for any labour
information. SQL in the optimisation phase is useful to arrange in an ascending order the resulting costs so that
selection of minimum cost chromosomes over all generations is enabled. The developed relational database enables
the designed system to be more flexible in terms of adapting any number of processes, any number of shifts, any
number of crews, and any number of workers. By having lists (workers, crews, or processes lists) that cross-
reference each other, a flexible scenario can be built and created.
3.2 Modelling Phases of a 3D Simulation Model
The methodology used to develop a 3D simulation model starts with identifying the logic of labour-driven
processes. Many structured interviews with production managers and senior workers beside a number of on-site
visits are conducted, to capture the hierarchy of such production system and to identify the interchanging
relationships among its components. The logic of production processes was identified and a flowchart was prepared
by Al-Bazi, et. al (2009). Figure 2 shows the development phases of the 3D simulation model.

FIG. 2: development phases of a 3D simulation model
After identifying the logic of the processes, the simulation modelling process starts by translating the static version
of the process logic into a dynamic simulation model. ARENA SIMAN language is used to enable such translation
by adopting Discrete Event Simulation (DES) concept. The resulting model consists of simulation blocks that linked
with each other by using smart links. Those blocks involve decision, process, assign and other useful modelling
blocks that ARENA software provided. The visualisation of simulation model starts by using the available modules
from the 2D library. Such animation gives a 2D representation useful to identify the verification of the model. The
3D visualisation is developed using advanced 3D modules which 3D-ARENA provided. The purpose of such
modelling is to provide a navigation tool that enables users or production managers to explore the virtual models
components and to see whether it has been successfully represent the real world manufacturing system or not.

3.3 Developing an Intelligent Searching Algorithm: Genetic Algorithms
Genetic Algorithm (GA) works with a population of solutions rather than a single solution. In this problem, an
optimal allocation plan needs to be chosen from a large pool of plans. Genetic Algorithms deal with this sort of
searching rule as the population of crews can be coded in terms of chromosomes, and then each chromosome can
call the desired crew by referring to its index. The main innovative contribution of a GA is the novel construction of
a neighbourhood based on natural selection principles, lafsson, et. al (2002).
210
9
th

3.3.1 Multi-Layers Chromosome Engineering
For a Multi-Layers chromosome, all genes have given sequential numbers, started from 1 to the number of the last
gene allocated in the last layer of the chromosome. This sort of Multi-Layers chromosome structure is proposed and
developed in order to have less parameter to pass through while coding the program and to make comparison easy.
See figure 3

FIG. 3: mutli-Layers chromosome structure
The idea behind designing a Multi-Layers chromosome is to present how different attribute labour inputs can be
stored in a chromosomal structure design. This structure is useful to present different sets of inputs in terms of
multiple layers so it can coded easily and presented better. The nature of inputs for the SIM_Crew model drove
the developed chromosome to adopt Multi-Layers structure rather than a conventional form. Each set of inputs
involved production process ID, number of crews available to carry out each process and the shift type. For each
process, possible alternative crews involving all required skilled of workers are stored into a pool of crew
alternatives. The first layer of the chromosome involves all possible daytime crews. The second chromosome layer
is assigned to accommodate nightshift crews. The designed chromosome enables Structured Query Language (SQL)
to guide the searching of any process to find a feasible allocation plan. Each process may involve more than one
crew in which any of them is able to carry out the process with different crew formation and within a different
processing time. Each process can be carried out using more than one working shift to satisfy commitments with
clients, so not all processes are necessarily to have the same number of working shift processes. The developed
Multi-Layers chromosome enables each set of data relating to a working shift to be placed into a specified Layer.

3.3.2 Population Initialization
The initial population, which is then evolved by GA, should, if possible, be well spread through the search space, so
that it is well sampled. Random sampling is used to generate the initial population.
As one of the random sampling techniques, Monte Carlo (MC) simulation is used to generate crews indices. For
each gene, an integer random number represents position of the crews alternative is generated to select randomly
the crews alternative of each process.

3.3.3 Objective Function of SIM_Crew
The purpose of allocating crews is to optimise the manufacturing system performance; thus, manufacturing system
performance such as manufacturing time, production cost obtained from a running simulation is used as the fitness
211
9
th

value within the GA operation. The objective function is applied to evaluate the total resource costs. The equation
used to calculate such objective function is:
(1)
Where:
: cost objective function values obtained by evaluating chromosome i
n : the number of labour-driven processes

i
BRC : incurred cost per hour when using a labourer for set of solution i.
i
IRC : incurred cost per hour when labourer is idle for set of solution i
i
RCPU : incurred cost per use of fixed or physical resource for set of solution i,
senior skilled bonus can be considered in such cost.
In some situation, there is a trade-off exists between idle time and cost (El-Rayes, et. Al 2001). In this study, only
direct cost is considered as a substantial cost.
Calculation of Fitness Function
For minimisation type problems, modifying objective function is required to give high weights for minimum costs to
be selected for further evolution. The modified objective function is called fitness function, the fitness function
expression is:
(2)
Where:
Fitness function value for chromosome i
Largest cost in population i

3.3.4 Development of the Genetic Algorithms Operators
For the allocation system being developed, GA operators have developed to solve this type of allocation problem.
Selection, crossover and mutation strategies are developed to achieve a better search for promising solutions. Al-
Bazi, et. al (2009).
The Proposed Class Interval Selection Rule (CISR)
In this proposed rule, only the promising chromosomes with least costs or higher fitness functions will be considered
as a potential improvement vehicle. The proposed selection rule is named class interval rule, which is developed
to provide the promising chromosomes with higher probability of selection to produce good solutions. The main
concept of this selection rule depends on constructing a class interval which is used in descriptive statistics.
Repetition of any generated chromosome is not allowed, so all generated chromosomes should be unique all over the
evolution process,
Probabilistic Dynamic Crossover (PDC) Strategy
The crossover operation in a conventional GA is based on the exchange of genes between two fixed length
chromosomes when coding is applied for chromosomes. To crossover genes in the chromosome, (0-1) variates
should be generated for each gene in the Multi-Layers chromosome. This type of exploration investigate all active
genes (occupied genes by scheduled crew with a shift) for more randomness. A random number is then generated to
exchange genes after satisfying a certain condition. In this strategy, random numbers are generated to be associated
with each gene at each layer, if the gene is vacant for a reason (no applied shift if there is no crew) then the
generated random number will be discarded to skip to the next gene.

212
9
th

Probabilistic Dynamic Mutation (PDM) Strategy
To avoid local maxima and to randomise the searching process, a modified mutation process is developed to swap
the gene within a chromosome with its available set of alternatives. This strategy is similar to PDC but the only
difference is that Monte Carlo (MC) sampling applies to search stochastically for a crew within each assigned pool
of crews. Each offspring (an individual chromosome) is randomly selected and genes are mutated vertically with its
set of alternatives using Monte-Carlo sampling.

4. CASE STUDY
A real life case study for one of the largest sleeper precast concrete manufacturer in the UK was developed. The
purpose of developing this case study is to identify the performance of the proposed allocation system. In sleeper
manufacturing system, the production process involves using of a wide range of different resources, including
labour, equipment and materials. Each production section has two labour-intensive production lines; shared
resources (workers and machines) are used to carry out job/activity on each production line alternately. Eight
processes including curing process are applied on the sleeper product being produced to deliver the final product.
Similar labour-intensive processes are adopted at each production line, processing time depending on the type of
product being produced at any production line.
A simulation model is developed using ARENA simulation software to simulate the production processes of the
sleeper concrete manufacturing system. All relevant data was collected by conducting onsite visits, using flowcharts,
interviews and stopwatch technique as relevant data collection techniques. More detail about crew formation data,
worker details are confidential and has not provided in this study. See figure 4 for the 3D animation of the sleeper
manufacturing system considered in this study.

FIG. 4: snapshot of the sleeper precast manufacturing system
In figure 4, all shared resources such as casting machines, run strand wire car and stress machine are animated to
show how such productive resources can be shared to carry out a process visually.

4.1 Experimental Analysis and Evaluation
The experimentation part of this study involves allocating possible crew alternatives to production process in order
to identify minimum allocation cost. In this case study, Multi-Layers chromosome structure is developed to store
crew alternatives, processes and shift patterns. The parameters used in GA are carefully selected by design of
experiments. It has been noticed that manipulation of parameters makes no significant differences in terms of
solution quality. The best parameter settings are thus identified as follow: the population size is found to be 20. The
gene crossover probability is identified to be 0.70 and gene mutation probability is 0.90. Number of processes is 28
processes and 66 resources (workers and machines) are utilised in this job-shop. Two working shifts are applied in
section 1 and one shift is in section 2. The stopping condition is satisfied when there is no reduction in the resulting
cost for five consecutive generations (100 chromosomes). A number of scenarios are developed to evaluate
213
9
th

performance of various allocation plans. The resource allocation cost is selected as a criterion of such scenarios. The
first scenario is Before Cost Drop in which the allocation plan before the first cost drop is identified. The
allocation plan obtained after the cost drop is called After First Cost Drop scenario. The current adopted scenario
is called As-Is. The scenario with the minimum allocation cost is called Best Allocation Plan followed by
Worst Allocation Plan obtained by adopting maximum allocation cost. Figure 5 shows the reduction in cost using
SIM_Crew allocation system.

FIG. 5 cost reduction using SIM_Crew system
It has been noted that the results tend to be stable after 50 generations; a significant reduction in crew allocation cost
is obtained. This significant reduction occurred early at the beginning of the generations and after generation 20.
This reduction took place because GA operators have successfully explored more solutions that are promising and
provided the required randomness in the mentioned generations. The PDM operator played a vital role in bouncing
the solution out of local minimum traps. The high probability of gene mutation kept crews manipulation of most
processes (genes) active, which eventually can identify more promising solution.

4.2 A Comparison Study between All Scenarios: Process-Waiting Times
A comparison study is conducted to identify the effect of adopting different allocation plans on the crew allocation
cost. Average process-waiting time is selected as a substantial factor to compare these allocation plans in order to
come up with the best plan. Figures 6 and 7 show comparison of average process-time obtained after running each
scenario.

FIG. 6: Comparison of average process-waiting
time achieved in production section 1

FIG. 7: Comparison of average process-waiting
time achieved in production section 2
214
9
th

Figures above reveal that the best allocation plan scenario achieved minimum average process-waiting time. The
highest process-waiting time is achieved by running the worst allocation plan in section 2. In addition, before
first cost drop has a process-waiting time close to the worst allocation plan scenario. The minimum process-
waiting time obtained in both sections led to a minimum crew allocation cost.
The As-Is scenario considered as the second best scenario, the process-waiting time obtained using this scenario
was the second best waiting time. It has been concluded that idle time has a significant impact on the total crew
allocation cost as idle time is considered as a substantial factor in the total allocation cost function.

5. CONCLUSION AND FUTURE DEVELOPMENT
The proposed allocation system SIM_Crew showed that it was possible to minimise crew allocation cost by
adopting an innovative allocation system. The developed simulation-based GA can provide an optimised crew
allocation plan to any labour-intensive industry. Probabilistic dynamic crossover and mutation operators were
developed successfully to explore the solution space by avoiding local minima solutions. After running SIM_Crew,
the results showed a significant cost reduction by allocating the proper crew to the right process and take into
account minimising process-waiting time. The developed operators played a vital role in adding more randomness to
the searching process. The new concept of using a GA in the crew allocation process and the developed Multi-
Layers chromosome application has been proven as a sophisticated and advanced technique through this case study.
Different levels of priority for each production process can be included as a future work when designing the
chromosome. Multi-objective optimisation is also important in order to model this type of allocation problem.

6. REFERENCES
Al-Bazi, A. and Dawood, N. (2009).Decision support system for pre-cast concrete manufacturing planning: An
innovative crew allocation optimiser. The 2009 CSCE International Conference on Computing in Civil
Engineering, St. John's, Newfoundland & Labrador, Canada.
Azadeh, A., and Ghaderi, F. (2006). A Framework for Design of Intelligent Simulation Environment. Journal of
Computer Science 2 (4): 363-369, 2006
Dawood, N, Ahmed, R. and Dean, J. (2007). Modeling of Precast Concrete Production Operations and Innovations:
A Simulation Approach. Manubuild Conference, Rotterdam, 25-26 April.
El-Rayes , K., and Moselhi, O. (2001). Optimizing resource utilization for repetitive construction projects. Journal
of Construction Engineering and Management, Vol. 127, No.1, pp. 18-27
Ipsilandis, P.G. (2006). Multi-objective optimization in linear repetitive project scheduling. Operational Research.
An International Journal. 6(3): 255-269.
Li, H., Love, P., and Ogunlana, S. (1998). Genetic algorithm compared to nonlinear optimization for labor and
equipment assignment. Building Research and Information, 26(6): 322-329.
Lu, M. , and Lam, H. (2005). Optimised concrete delivery scheduling using combined simulation and genetic
algorithms. In Proceedings of the 2005 Winter Simulation Conference, M. E. Kuhl, N. M. Steiger, F. B.
Armstrong, and J. A. Joines, (Eds.) 2572-2580. Piscataway, New Jersey: Institute of Electrical and
Electronics Engineers, Inc.
lafsson, S. and Kim J. (2002). Simulation Optimisation. Proceedings of the 2002 Winter Simulation conference
Pentico, D.W., 2007, Assignment problems: A golden anniversary survey, European Journal of Operational
Research, Vol.176, No.2, pp.774-793
Watkins, M., Mukherjee, A., Onder, N., and Mattila, K. G. (2007). Understanding Labour Productivity as an
Emergent Property of Individual and Crew Interactions on a Construction Site. In Processings of the IGLC-
15, July 2007, Michigan, USA: 400-405.
215
9
th

Marsono, A. , Masine Md. Tap, Ching N. S. and Makhtar, A. M. (2006). Simulation of Industrialized Building
System Components Production. In Proceedings of the sixth Asia-Pacific Structural Engineering and
Construction Conference (APSEC 2006), Kuala Lumpur, Malaysia
Marzouk, M. Hisham Zein El-Dein, and Moheeb El-Said. (2007). Application of Computer Simulation to
Construction of Incremental Launching Bridges. Journal of Civil Engineering and Management, Volume
XIII, Number 1, pp. 27-36.
Moselhi, O. and Alshibani, A. (2007). Crew optimization in planning and control of earthmoving operations using
spatial technologies. IT Construction, 12: 121-137.
Nassar, K. (2005). Evolutionary optimization of resource allocation in repetitive construction schedules, IT
Construction, 10: 265-273.

216
9
th

INTEGRATION OF URBAN DEVELOPMENT AND 5D PLANNING
Nashwan. Dawood, Professor
Centre for Construction Innovation & Research, Teesside University, UK
N.N.Dawood@tees.ac.uk
Claudo. Benghi, Dr
Built Environment, Northumbria University
Thea. Lorentzen, and Yoann. Pencreach
FORUM8, Japan
ABSTRACT: 5D planning is the process of integrating 3D models of buildings or infrastructure projects with
construction activities and cost planning. The process has been used successfully to rehearse the construction
process and indentifying hot spots and process clashes prior to site construction activities. Previous research
(Dawood, 2008) has concluded that at least 7% of construction time and cost can be reduced if the technology has
been deployed at an early stage of construction process. However, construction planning within the context of urban
development not fully exploited and tools and methods for synchronising and rehearing multiple construction
planning within urban setting and identify, for example, traffic congestions and other environmental issues that can
be affected with construction processes and nearby sites. In this context, the aim of this paper is to deploy and
develop 4D process and technology for urban planning and construction. The objective is to rehearse construction
processes for urban construction which can involve a multiple of construction projects, both buildings and
infrastructure can be constructed concurrently and can cause massive disruption and congestion at urban scale.
Traffic management and flow can be incorporated within the urban simulation and therefore congestions caused by
multiple construction sites can be identified and resolved before construction starts.
This paper presents a framework and tools for rehearsing multiple construction projects that was developed to
identify issues and hot spots at urban scale. The paper also present initial results of integrating Uc-win/Road visual
urban planning with nDCCIR 5D planning tool. A simple case study was used to demonstrate the technology.
KEYWORDS: 4D planning, Urban Planning, Virtual Reality

1. Introduction to nD planning and urban developments
Recent work in urban planning, developments and transportation visualisation is evolving from a focus on how
projects look to a desire to see how they actually work, i.e. the interaction between different objects and the
influence of different risk factors on urban developments. As many walk-through simulation methods focus on a
scenes aesthetic qualities, there has been a growing need for visualization of processes. As such, developments in
virtual reality simulation come as a result of an increasing recognition of the value of visualization for representing
not just infrastructure, but operations and in particular construction.
Virtual reality is a technology that enables users to interact with a simulated world. When applied to transportation,
VR has the potential to not only model traffic flows; it also allows for construction simulation in the form of nD
modelling. Observing built and natural environments from the construction team provides a viewpoint that is often
ignored in traditional planning methods. Particularly for cases where construction can affect traffic flow in an urban
scene, health and safety issues, portfolio investment and others.

The purpose of this system development is to implement 5D planning so that it can serve as a common visual and
experiential language for presenting, discussing, and ideally improving civil engineering and building construction
projects. The UC-win/Road software was first launched in 2001 and initial versions were primarily used as tools for
visualizing alternative road designs. With an engineering platform, the program continues to be used as a visual tool
for engineers to discuss traffic flows, road alignments and land-use issues. Recent versions have been adapted to
include 5D models of construction projects, animated traffic and human characters. The following section introduces
the concept of 5D models and urban development and the integration of these tools is explained and discussed.
217
9
th

1.1 5D project control
Multi-dimensional project control, also known as 5D Project Control, consists of enriching 3D model-based
construction information with data from other domains. In current literature, the 4th dimension is always referring to
time/schedule information while the 5th is commonly considered to be cost but may be related to other domains
such as risk, task ownership, products durability, etc.
4D construction process ideas have been developing for more than ten years by many academics and research
groups (McKinney, K. et al., 1996). 4D modelling is often related to advanced visualization techniques; the first
achievement of 4D modelling was the 3D visual animation of construction sequences through time allowing an
effective representation of inherently abstract scheduling data which held by Gantt charts in a inexpressive form.
Although not standardized or widespread on the market, 4D modelling traditionally refers to the ability to perform
such a visual rehearsal of construction planning, often according to the workflow presented in figure 1.

FIG. 1: Process flow for traditional 4D rehearsal

In the represented process Planning and 3D drawing are connected because the CAD model often needs to be
organized accordingly to the determined planning sequences before the actual linking process can begin; when this
is the case 3D objects are usually grouped in CAD layers accordingly to the planned activities and finally activity
names or codes are mapped to the corresponding layers. This process requires ad-hoc and time consuming drafting
and linking sequences.
The availability of integrated engineering design tools, including Building Information Modelling applications,
provides a unique authoritative repository for the storage of all the design information needed for the definition of
the construction programme. Exploiting such information, a paradigm shift is envisioned to use extended 5D tools
not just for programme but for integrated programme development (Tse et al. 2005)

FIG. 2: Process flow of theoretical integrated construction management
218
9
th

The present paper presents the rationale and half-term results of a two year programme aimed at developing methods
and custom IT tools to exploit the potential of such data repositories in the development of cost-aware schedule
programmes in the Oil and Gas industry.
1.2 Urban planning using VR
Virtual modelling of urban planning can provide architects, planners, investors, and managers a collaborative,
future-oriented tool. Current digital urban modelling projects such as Virtual Los Angeles and Virtual Kyoto are
especially focused on bringing together resources and processes to provide aid in disaster management planning. For
example, the Los Angeles model aims to provide information not only to urban planners but also to emergency
response teams and government entities (Virtual Los Angeles). The ability to model several processes at the same
time allows for scenarios such as building evacuations or traffic accidents to be studied prior to their occurrence
(i.e..real-time traffic input for evacuation during hurricanes.)
In this sense, virtual project modelling can provide a platform for users to be not only informed, but also engaged in
project plans. Ideally, project time can be reduced if stakeholders are involved and less resistant throughout the
approval process. Also, with a variety of participants simultaneously negotiating a plan, earlier identification of
problems and objections as well as better informed decisions can assist in reaching project goals (Walker, 2002).
When applied to urban spaces, virtual reality has the potential to create a platform that allows users to not just view,
but interact with a highly accurate visualization of transportation agents and built environments. Proposals can be
juxtaposed against existing structures. The current VR system has been used to create a digital model of Phoenix,
Arizona with the goal of goal of mitigating urban sprawl problems by facilitating smarter planning. Similarly, the
program is being used to communicate with citizens about the construction of a new LRT route and urban renewal
program in Sakai City, Japan. The VR environment is able to reproduce different time intervals and scenarios,
allowing presentations and interactions to encompass changes in space, time and activity. The potential development
of these tools with multi-dimensional project control is fully explained in the next section.
2. RESEARCH THEMES
The use of multidimensional project control systems (5D Planning) has proven benefits in a number of different
indicators of the procurement process value: On site productivity, Team integration, Client satisfaction, Planning
efficiency, Number of rework, Communication, H&S related risk and Construction time; the advantages of all these
sum up to an estimated 7% in terms of expenditure savings (Dawood and Sikka, 2006).
Current multidimensional research roadmap at CCIR (see Figure3) focuses on the development and evaluation of
integrated tools across all of the procurement process; the vision is to develop a modular and integrated platform
taking advantage of Rapid Application Development frameworks (RAD), available open source software
components and open formats for interoperability and data exchange to extend the role of available tools in the
direction of a tighter integration of design, planning, construction and urban planning. The next paragraphs will
describe the objective of the active research topics.

Procurement
Stage
Functionsofcurrent
tools
Activeresearchtopics Furtherdevelopment
Design Semiautomatedcostandplangeneration
Planning Schedule
rehearsal
Costintegration
Scenarioevaluation
Integratedplan
development
Intelligentprogrammeoptimization
Spaceresourceoptimization
Integrationwithurbanplanning
Construction Schedule
communication
Programme
controland
adjustment
Subcontractorsschedule
synchronization
Identifytrafficcongestions
Operations H&SControlandtraining
FIG. 3: Medium term multidimensional research roadmap at CCIR
219
9
th

2.1 Previous development
The development of the new platform is based on the experience gathered in almost a decade of research in the field
of multidimensional construction management including the development of VIRCON (Dawood and Scott, 2005)
and 4DCCIR (CCIR, 2007). Although available tools already allowed performing plan rehearsal and communication
a new framework has been designed to incorporate the experience gathered during the projects and to allow the
implementation of the functionalities mentioned in the next paragraphs and improve scalability for further
extensions.
2.2 Cost and integration
The availability of Cost information in the 5D model is a major step towards the complete integration of most
relevant project data. For the purpose of multidimensional project control, two types of cost are identified:
Costs of resources required for the construction, including labour and material resources; and
Costs associated with building components and materials.
The former are usually managed in Project Management (PM) application in association with project resources; the
latter could be obtained merging quantity information either available in the engineering information or extracted
from 3D models with unit costs available in material and cost libraries.

FIG. 4: Components and processes cost management

To integrate PM cost management, project management data importers had to be extended to read resources and
their usage information from Primavera P3 and P3e and Microsoft Project 2003 and 2007. Cost information from
these applications can be referred to model components through the links that are established with the activities for
4D rehearsal, so that no additional task is required from users to link them.
Development of the cost libraries integration is currently under development along with the interfaces required to
specify engineering information required to correctly specify quantities and related unit costs. The planned
development though includes the ability to perform an automatic identification of the associated costs in the library
where appropriate engineering information is available within the 3D model (i.e. for BIM models).
Integrated cost information allows project managers to
visually assess cost and cash-flow information in association with the project stages;
improve assessment of the financial relevance of building components and processes through colour-
coding;
improve communication with clients; and
Allow budget control and management.

220
9
th

3. SCENARIO EVALUATION
One of the limits identified by previous research was the need to continuously switch between different applications
to improve the schedule as rehearsal helped identifying errors in the programmes. None of the reviewed
multidimensional tools offers the ability to modify schedule data can only import it from planning applications.
Adding the ability to modify programmes interactively is not a trivial task because programme activities have
repercussion on each other when activities dependencies are defined. A scheduling solution engine had to be
developed to evaluate the changes deriving from changing activities dates or durations and their dependencies.
Compared to well establish commercial applications, the developed schedule solution engine is not complete;
however it allows us to perform integrated what-if scenario evaluation within the interface thus saving much time
and immediately seeing the consequences on each programme variation. The application keeps track of all the
modifications the user performs on the tasks and relations so that the same changes can then be applied in the
original project management environment for a data sanity check.

3.1 Urban planning and integration
The micro simulation .xml file format allows for position and movement information for various urban elements to
be displayed together in the same space. Several simulations can be opened at the same time and a playlist manages
their relative positions in time. For example, construction processes, traffic flows, pedestrian movement and
environmental / lighting changes can be shown in the same VR model. The behaviour of these elements can also be
recorded and played back, also allowing for fast forwarding, pausing and jumping though time. Logs of position
information, etc. can be exported for later analysis. When several simulations are shown simultaneously, the
interplay of elements within one urban world can be tested so as to avoid collisions in real life. As the micro
simulation format is an open one, we expect that behaviour of other location-specific elements, such as
environmental disasters or material cost information will be added by different users. Main features of the micro
simulation include:

Playlist
Manages the relative position in time amongst several simulations.
Sets time unit conversion for each simulation (e.g. 1 day played in 1 second)
Playback
Support of various time units (nsec to year). By default the original time unit is converted to second
for playback.
Standard playback support: Play, Stop, Fast forward, Pause, Jump

3.2 Current development
Ongoing development also includes the ability to keep track of updated progress/activity information and to
compare it with the saved baseline to achieve a comprehensive progress control system allowing the Project
Manager to:
visually compare the expected status of the construction with the current on-site situation at any given
moment in order to correctly assess activity completion percentages;
rehearse the execution of the tasks due in the following weeks in order to identify ahead previously
undetected conflicts;
promptly react to any programme change being able to improve assessment of the consequences
deriving from delayed activities, on site problems or programme variations; and
Effectively manage task reallocation depending on budget constraints and policies.
Integration with visual urban planning to identify traffic congestion and rehearse of construction plans
of multiple projects.
221
9
th

4. IMPLEMENTATION
The discussed project has been developed for the Windows platform on the Microsoft .Net Framework as a multi-
tier application according to the scheme reported in figure 5.

FIG. 5: Development framework components grouped by tier
222
9
th

FIG. 6: A screenshot of the application in project control mode within urban setting with

223
9
th

5. CONCLUSIONS
Although formal evaluation of the developed tool is still to be undertaken, significant interim conclusions can be
drawn:
data import functionalities from PM applications and Open 3D formats have been tested extensively
and have proven to provide all the required information;
public domain software development kits for the gaming industry along with the diffused availability
of accelerated 3D graphic adapters provide suitable frameworks for the development of 3D and
possibly Virtual Reality visual environments;
The development of the schedule solution engine allowed us to include methods for the rapid
evaluation of alternative programmes within the rehearsal interface and could ultimately result in the
development of an integrated multidimensional programme development environment.
the use of publicly available Open Source components in Rapid Application Development
environments allows effective application development of articulated frameworks;

6. REFERENCES
4DCCIR, CCIR, (2007). Available at: http://www.idproplan.co.uk/4DCCIR.html [Accessed July 10, 2008].
AGA Treeviewadv, (2007), Available at: http://sourceforge.net/projects/treeviewadv/ [Accessed July 10, 2008].
Al-Kodmany, K. (1999). Using visualization techniques for enhancing public participation in planning and design:
process, implementation, and evaluation, Landscape and Urban Planning, 45, pp. 3745.
Bentley, (2007), Engineering Project Schedule Simulation. Available at: http://www.bentley.com/en-
US/Products/ProjectWise+Navigator/Schedule+Simulation.htm [Accessed July 11, 2008].
Bentley, (2008), Bentley Acquires Common Point to Mainstream Construction Simulation. Available at:
http://www.bentley.com/en-US/Corporate/News/Quarter+2/Common+Point.htm [Accessed July 11, 2008].
Cadalyst, (2007), Autodesk Acquires NavisWorks. Available at:
http://aec.cadalyst.com/aec/article/articleDetail.jsp?id=430659 [Accessed July 10, 2008].
Dawood, N. and Sikka, S., (2006). The Value of Visual 4D Planning in the UK Construction Industry. In
Intelligent Computing in Engineering and Architecture. pp. 127-135.
Dawood N, Scott D., (2005). The virtual construction site (VIRCON) tools: An industrial evaluation. Available at:
http://www.itcon.org/cgi-bin/works/Show?2005_5 [Accessed July 10, 2008].
Dockpanelsuite, (2007). The docking library for .Net Windows Forms, Available at:
http://sourceforge.net/projects/dockpanelsuite/ [Accessed July 10, 2008].
Laiserin, J., (2008). Vico Virtual Construction Suite 2008 (Cadalyst Labs Review) - CAD Management. Available
at: http://management.cadalyst.com/cadman/article/articleDetail.jsp?id=526884 [Accessed September 11,
2008].
LZMA SDK (Software Development Kit), (2007), Available at: http://www.7-zip.org/sdk.html [Accessed July 10,
2008].
McKinney, K. et al., (1996). Interactive 4D-CAD. In Computing in Civil Engineering. Atlanta, GA: Georgia
Institute of Technology, pp. 383-389 .
Osborne A, (2007). Commonpoint data integration tools and methods. Personal interview with Commonpoint Inc.
Vice President Sales and Marketing
Stowe, K., (2008). Autodesk - Revit Extension. Telephone interview with Autodesk In. Construction Business
Development Manager
224
9
th

Tse T K, W.K.A.A.W.K.F., (2005). The utilisation of building information models in nD modelling: A study of
data interfacing and adoption barriers. Available at: http://www.itcon.org/cgi-bin/works/Show?2005_8
[Accessed July11, 2008].
Walker, D., (2002). Visualization as a Common Language for Planning: Good Practices, Caveats, and Areas for
Research, TR News, Going Public: Involving Communities in Transportation Decisions, Transportation
Research Board of the National Academies, 220, pp. 7-11, 2002.
Virtual Los Angeles, University of California, Los Angeles
http://www.ust.ucla.edu/ustweb/projects.html
Virtual Kyoto
http://www.geo.lt.ritsumei.ac.jp/uv4w/frame_e.jsp
http://www.ritsumei.ac.jp/eng/newsletter/winter2006/gis.shtml
Center for Advanced Transportation Technology, University of Maryland
http://www.catt.umd.edu/research/index.html
www.openmicrosim.org

225

SIMULATION AND ANALYSIS

A Simulation System for Building Fire Development and
the Structural Response due to Fire----------------------------------------------------------229
Zhen Xu, Fangqin Tang and Aizhu Ren

Physics-based Crane Model for the Simulation of Cooperative Erections-----------237
Wei Han Hung and Shih Chung Kang

Interaction between Spatial and Structural Building Design:
A Finite Element Based Program for the Analysis of
Kinematically Indeterminable Structural Topologies-------------------------------------247
Herm Hofmeyer and Peter Russell

Virtual Environment on the Apple iPhone/iPod Touch-----------------------------------257
Jason Breland and Mohd Fairuz Shiratuddin

3D Visibility Analysis in Virtual Worlds: The Case of Supervisor---------------------267
Arthur van Bilsen and Ronald Poelman

Evaluation of Invisible Height for Landscape Preservation
Using Augmented Reality-----------------------------------------------------------------------279
Nobuyoshi Yabuki, Kyoko Miyashita and Tomohiro Fukuda

An Experiment on Drivers Adaptability to Other-hand Traffic
Using a Driving Simulator----------------------------------------------------------------------287
Koji Makanae and Maki Ujiie

C2B: Augmented Reality on the Construction Site----------------------------------------295
Lon van Berlo, Kristian Helmholt and Wytze Hoekstra

Development of a Road Traffic Noise Estimation System
Using Virtual Reality Technology-------------------------------------------------------------305
Shinji Tajika, Kazuo Kashiyama and Masayuki Shimura

Application of VR Technique to Pre- and Post-Processing for
Wind Flow Simulation in Urban Area--------------------------------------------------------315
Kazuo Kashiyama, Tomosato Takada, Tasuku Yamazaki, Akira Kageyama,
Nobuaki Ohno and Hideo Miyachi

Construction Process Simulation Based on Significant Day-to-day Data-------------323
Hans-Joachim Bargstdt and Karin Ailland

Effectiveness of Simulation-based Operator Training------------------------------------333
John Hildreth and Michael Stec
9
th

A SIMULATION SYSTEM FOR BUILDING FIRE DEVELOPMENT AND
THE STRUCTURAL RESPONSE DUE TO FIRE
Zhen Xu, Doctoral candidate,
Department of Civil Engineering, Tsinghua University, Beijing, P.R. China;
martin31567@gmail.com
Fangqin Tang, Postdoctoral,
tangfangqin00@mails.tsinghua.edu.cn
Aizhu Ren, Professor,
raz-dci@tsinghua.edu.cn
ABSTRACT: The software on simulation of building fire and the software on simulation of structural response due
to fire both have good performance in their respective fields, but they are not good at the synchronous simulation
between building fire and structural response due to fire. In this paper, the synchronous simulation above is made in
the virtual reality system based on Vega. This system is a post-processing platform based a fire simulation software
FDS and a FEM software MSC.MARC. FDS and MARC provide scientific results for synchronous simulation, while
this virtual reality system shows these results in a visual and lifelike way. This system reveals the link between
building fire and structural fire response. In virtual training, this system can help people judge the structural safety
by the situation of building fire.
KEYWORDS: Building fire, Structural fire response, Synchronous simulation
1. INTRODUCTION
Fire is one of the most dangerous disasters to destroy the buildings. The fire safety of building structures is therefore
paid more and more attention to. With the application of the virtual reality technology in more and more fields,
many scientists have studied on the simulation of building fire, such as fire numerical model (Fu Zhuman and Fan
Weicheng, 1996; Yao Jianda et al., 1997; H. Xue , et al, 2001), fire graphic technology(P Beaudoin, et al,2001) and
application of fire simulation(Wang Jian and Fan Weicheng, 1996; Jurij Modic, 2003). But there are very few
software systems that can simulate the building structural response due to fire with the development process of
building fire. In order to study the dynamic building response with the development of a building fire, a virtual
reality system was developed, which can simulate the building structural fire response synchronously with the
development of a building fire.
A variety of software has been developed in the area of fire simulation, such as FDS, PHOENICS and FLUENT
(Jiang Ling et al., 2009). The structural response under a fire can be calculated and simulated with a series of
software, such as ANSYS, ABAQUS, MSC series software and other well known FEM software.
The system developed by the authors is based on the fire simulation software FDS and the structural analysis
software MARC. The temperature scene is obtained from FDS, the structural fire response analysis results are
obtained from structural analysis by employment of MARC which reads the data file of the temperature scene as
input file to the software MARC. It is difficult to realize the simulation of the building structural fire response
synchronously with the development of a building fire in a virtual reality environment. The authors employed
different screens to display the simulation process of fire and smoke spreading and the deformation of the structure
associated with the fire development. Since the models are different in the simulation of building fire and the
simulation of structural fire response, it is difficult to access the two different models correspondingly.
The solutions to this are introduced in the following sections. The system architecture, the algorithms and the
development solutions are also presented in this paper.

229
9
th

2. SYSTEM DESIGN
This system was based on a well known VR software Vega, developed by Microsoft Visual C++ 6.0, in order to
accomplish the synchronous simulation between building fire and structural response. The system shows the process
of the dynamic structural response with the development of a building fire in dual windows as shown in Fig. 1,
while the users can walk through in two scenes. The dual windows can be switched from one window to the other.
For more realistic immersion, the building fire scene or structural fire response results are visualized in the full
screen.

FIG.1:Interface of system
FDS is adopted in this paper to simulate the temperature variation, the smoke spread in a building, while
MSC.MARC is adopted to analyze the structural fire response. The structural fire response is analyzed based on the
FDS results. To avoid effect of dimensions of models in different calculated environments, the dimensions of
building model in virtual environment are the same as the dimensions of model in FDS. The building model in
virtual environment is constructed by AutoCAD. In addition, a modelling software Creator, which matches software
Vega, is adopted to create structure models at different time. The results of FDS and MARC are used as input data
for this system. In this system, smoke control, deformation control and synchronous control are important steps for
the accomplishment of synchronous simulation, as shown in Fig. 2.

FIG.2:ystem flow
230
9
th

3. DEVELOPMENT SOLUTIONS
3.1 Smoke control
In Vega, the visualization of flame and smoke are accomplished by special effect module. The technology of flame
visualization is to render texture in cycles, and the smoke visualization is based on particle system (Wang Jingqiu
and Qian Zhifeng, 2001). In this system, flame and the part of smoke are realized by special effect module in Vega,
but the effect of smoke spreading is realized by gird controlling (Chen Chi et al., 2007).
In Vega, particle system is difficult to control directly. The system divides building into many girds and the smoke
density in each gird is calculated by FDS. In each gird, the effect of smoke is created by the textures rendered in
cycles and the original state is set to be closed. When the smoke density reaches the threshold in some girds, the
smoke state in this gird will be set to open and simulation will show smoke spreads to these gird fields, as shown in
Fig. 3. The general controlling method of smoke spreading in girds as show in Fig. 4, in which the numbers
represent different types of smoke density in girds and the type 4 means the smoke density is beyond threshold.

FIG.3: Smoke controlling flow FIG.4: Smoke controlling schematic diagram
3.2 Deformation control
This system shows the structural response under a fire only by structural deformation, not including the stresses of
structural member. In frame analysis of MARC, structural members are displayed in the form of line and the point
displacements at different time can be calculated. Similarly, this system adopts line to display structural members
and use point displacements to control structural deformation.
Deformation control is to construct structural deformation models at different time by MARC results. In this system,
models at different time are constructed by calling the function from Creator. Structural members are divided into
many small lines and MARC calculates and outputs the positions and displacements for the ends of these small
lines.
According to these positions and displacements, the system calls the function mgSetCoord3d() to draw the lines in
Creator and calls the function mgSetAttList() to set the gradual colour, which shows the value of displacement
(MultiGen-Paradigm Inc, 2001). The whole structural models of building at different time can be constructed in this
method, Fig. 5. The red intensity of the colour is proportional to the value of displacement.

FIG.5: Structural deformation model
231
9
th

3.3 Synchronous control in time
Vega applications include two parts: original setting and main loop (MultiGen-Paradigm Inc, 2004). In the main
loop, the 3D scene is rendered in real time. Synchronous simulation between building fire and structural fire
response requires a synchronous rendering of two scenes in the main loop. Two processes must be executed in the
same time-step, so that synchronous simulation of dynamic process can be achieved.
For the simulation of building fire, FDS outputs smoke density in girds in certain time-steps and system identifies
the girds that the smoke has spread to. In the main loop, the smoke states of girds are set as open at every time-step
if smoke density of these girds reach threshold. In this method, the simulation of building fire can be accomplished.
For the structural response under a fire, MARC outputs displacements at the same time-step as building fire and the
structural deformation models at each step are created by these displacements. In the main loop, the structural
deformation models are loaded into the related scene at every time-step. The simulation of structural fire response
therefore can be accomplished.
Although two simulations adopt the same time-step, the simulation is not precisely synchronous, because different
realization methods cost different time in two simulations. So time control is necessary. Function vgGetTime() in
Vega library can be used to get the time from the beginning of the main loop to the present. For two simulations, the
real time spent at each time-step can be gained by vgGetTime(). When the difference of two spent time is beyond
the threshold, fast simulation will pause to wait for slow simulation and keep in same step.
3.4 Synchronous control in space
In this system, synchronous walk-through is an important function. The user location keeps equivalent at the two
scenes in the virtual environment. To realize the synchronous walk-through, the settings of two scenes, objects and
motion models must be consistent and two scenes need to be driven on a same input device.
There are many important parameters in Vega, such as window, channel, scene, and observer. To realize the
synchronous simulation in two screens, these parameters are set in two different ways and two settings need to keep
consistent in each term. The dimension of building model is the same as the structural model and the locations
where two models are loaded in two scenes also keep consistent. In addition, the motion mode at different scenes
must be the same. In the process of simulation, users act as two observers at the same time and observe two
windows, walking in two scenes by one input device, as shown in Fig. 6.

FIG.6: Synchronous walk-through flow
232
9
th

4. AN APPLICATION EXAMPLE TO THE SYSTEM
A two-floor house made up of light steel frame is selected for an example for system application. The system
simulates a fire on ground floor caused by sofa in living room and the time of simulation is 15 minutes. Steels have
no any fire prevention measures, covered by a layer of gypsum board. To display the feature of structural
deformation obviously, the values of deformations are enlarged properly.
The synchronous simulation of building fire and structural fire response is showed as follows:

FIG.7:Structural deformation and fire situation (1 min)
Fire is small and few smoke overflows from windows on right wall. There is no deformation caused by fire.

Fire grows rapidly and plenty of smoke can be seen in top of firing rooms.
The structural deformation firing rooms get larger and some members with large deformation grow red.

233
9
th

Fire burns fiercely. Large number of smoke overflows from all windows and doors and flame can be seen in the window of firing
rooms. There have been severe deformation in some important members and the whole structure is in danger.
The conclusion can be found from Fig. 7 to Fig. 9 that structural deformation grows with development of fire, which
agrees with common sense. In addition, structural deformation concentrate in firing rooms, although smoke has
spread into many rooms. The distribution of deformation is consistent with the distribution of temperature. At 15
minute, the temperature distribution calculated by FDS is show in Fig. 10.

FIG.10:Temperature distribution in building by FDS
An image from inner of firing room is showed in Fig 11, which helps users study link of building fire and structural
deformation further.

FIG.11:Fire situation and structural deformation in firing room (10min)
In firing room, fire is fierce and smoke layer is so thick that wall and ceiling cant be seen. If simulation just has one
scene of building fire, the structural information cant be gained. But in synchronous simulation, structural
deformation can be observed clearly in structural deformation scene. Some beams, which are deformed severely, are
red thoroughly.
At this time, severe deformations have taken place in the structure of firing room and the possibility of local failure
is large. Facing this kind of fire situation, fire men should stop entering this room and evacuate from this room in
virtual training.
5. CONCLUSION
The system is developed for the display of fire development process and for the learning of the building structural
response rules. Though it is currently a prototype system it shows the application potential for the training of fire
fighting and the checking of the building fire resistance designs. The results show that the process of dynamic
structure response with the development of a building fire in a visual and lifelike way can be realized by
employment of the solutions presented in this paper.

234
9
th

6. REFERENCES
Chen Chi, et al.(2007). "A building fire simulation system based on virtual reality", Journal of Natural Disasters,
Vol.16, No.1, 55-60.
Fu Zhuman , and Fan Weicheng.(1996) . "Building fire simulation method and development", Exploration of Nature
Vol.15,No.1,28-33.
H. Xue , et al.(2001). "Comparison of different combustion models in enclosure fire simulation", Fire Safety
Journal, Vol.26, No.1, 37-54.
Jurij Modic.(2003). "Fire simulation in road tunnels", Tunnelling and Underground Space Technology, Vol.18,
No.5, 525-530.
Jiang Ling, et al.(2009). "A brief analysis in the current development of fire computer simulation technology", Fire
Science and Technology, Vol.28, No.3, 156-159.
MultiGen-Paradigm Inc. (2004) . "Creating Models for Simulations (version3.0.1) ", U.S.A.
MultiGen-Paradigm Inc. (2001). "Vega Programmers Guide (Version 3.7) ", U.S.A.
P Beaudoin, et al.(2001). "Realistic and Controllable Fire Simulation", GI 2001: Graphics Interface, Ottawa,
Canada, June.7-9, 159-166
Wang Jian, and Fan Weicheng.(1996). "Numerical simulation of fire process of multi-rooms", Journal of University
of Science and Technology of China, Vol.26, No.2, 204-209.
Wang Jingqiu, and Qian Zhifeng.(2001). "Fireworks simulation based on particle systems", Journal of Nanjing
University of Aeronautics & Astronautics, Vol.28, No.3, 166-169.
Yao Jianda, et al.(1997). "Applications of FZN model in building fire", Journal of University of Science and
Technology of ChinaVol.27, No.3, 304-308.

235
9
th
PHYSICS-BASED CRANE MODEL FOR THE SIMULATION OF
COOPERATIVE ERECTIONS
Wei Han Hung, PhD. Student,
Computer-Aided Engineering Group, Department of Civil Engineering, National Taiwan University.
magic@caece.net
Shih Chung Kang, Assistant Professor,
Computer-Aided Engineering Group, Department of Civil Engineering, National Taiwan University.
sckang@caece.net
ABSTRACT: Cooperative erections are often very critical in modern construction projects. The use of
visualization technologies to simulate cooperative erection activities prior to construction can not only help
isolate problematic areas and plan corresponding operation strategies but can also reduce the risk of
unnecessary incidents. A numerical model is required to accurately simulate cooperative erections and the
physical reactions of a crane and its rigging system. In this research, we modelled a crane using three
sub-modules: (1) suspension module, (2) lifted-object module and (3) manipulation module. This model was
implemented and tested in a dual-crane scenario in order to investigate its feasibility. We found that the
proposed physics-based crane model can help deliver a realistic and interactive visualization.
KEYWORDS: Cooperative, erection, physics model, game physics, crane
1. INTRODUCTION
The use of cranes is becoming more versatile and yet challenging in modern construction projects. The tasks
involved are usually very risky and hence require high accuracy (Chi et al., 2007). The rigging objects are
larger, heavier, and come in more varieties than can be lifted by a single conventional crane. Therefore,
cooperative erections have become a trend in modern construction projects. Since large cranes are not always
available and site rental fees are sometimes very high, the cooperative use of standard cranes available on-site is
a cheaper alternative (Ali et al., 2005). Recently, some researchers have utilized a series of computational
methods to solve the path finding problems involved with the operation of cooperative and multiple cranes (Ali
et al., 2005, Chen and Amin, 2007, Kang and Miranda, 2008). However, due to space limitations and the vast
costs of erection activities, it is difficult to verify and test erection strategies using practical experiments.
In the past, many investigators have employed graphical technologies to simulate crane activities. This is a
low-cost and effective method for identifying potential problems before real construction. OConnor et al.
(1994), Liu (1995), Amatucci et al. (1997), Bernold et al. (1997), and Stone et al. (1999) developed various
simulation and visualization tools for crane operations. Lipman and Reed (2000) used the Virtual Reality
Modelling Language (VRML97) to provide 3D web-based technologies for managing, accessing, and viewing
construction project information. Kamat and Martinez (2005) automatically generated a dynamic 3D animation
of construction processes using discrete-event simulation DES tools. Huang and Gau (2003) designed an
interactive visual simulation on a cluster of desktop computers for mobile crane training. Kang and Miranda
(2009) developed a numerical method required for automatically simulating and visualizing detailed crane
activities.
Recently, some investigators and companies have introduced the use of physics engines to increase the accuracy
of crane simulations. Chi et al. (2007) proposed a physics-based simulation for the manipulation and
cooperation of cranes. Simlog (2007), CMLabs (2007), and the GlobalSim (2007) corporation developed
training simulators with real-time physical behaviours and a realistic rendering of scenes. These works have
demonstrated the effectiveness of integrating graphical technologies and physics engines, thereby increasing the
fidelity of crane simulations.
In this research, we would like to take this progression a step further. We would like to develop a more complete
and systematic physics model for cooperative crane operations. The main purpose of the research is to develop a
generalized physics-based modelling method for simulating cooperative erections. The latest game and physics
engines are used in the system to generate accurate and realistic crane simulations in real time. A suspension
module, lifted-object module, and a manipulation module were developed for simulating cranes, including the
rigging system (the crane), moving platform (mobile crane), lifting object, and also, lifting devices. The
developed model is able to dynamically simulate object attachment and detachment due to the separated
237
9
th
modules. An example case of a dual-crane scenario was used to test and investigate the feasibility of the
proposed model and demonstrate the results of the simulation.
2. MODELLING USING MULTYBODY DYNAMICS
We modelled a crane in multibody dynamics to simulate its physical behaviours, that the objects are composed
of interconnected rigid bodies and joints (Erleben et al., 2005). A rigid body, such as a wheel, cabin, hook, or a
lifting object, is called an actor in this research and their motions are constrained by the joints. In order to more
realistically simulate the physical behaviour of crane operations, we introduced springs and dampers into the
joint model. Four types of joints are used in this research: spherical, revolute, prismatic, and distance joints.
Descriptions of the elements of the physics model (actor and joint) and their representative symbols are shown
in Table 1. Detailed equations of multibody dynamics and their solver can be found in literature (Chi, 2007a,
Erleben et al., 2005). The joints are able to be dynamically attached and detached to simulate the linking and
releasing behaviour of rigging objects. Therefore, a continuous erection activity can be simulated.
TABLE. 1: Description of physics model elements and their representative symbols used in this paper (Chi et
al., 2007, NVIDIA, 2008).
Physics model element Symbol Description
Actor

A representation of a rigid body; an idealization of a solid body of finite size and is
un-deformable.
Spherical joint

A spherical joint is characterized by the fact that two points, one from each actor, are
always connected to each other. This means that there are three constraints and equality
in the x, y, and z dimensions of the two points.
Revolute joint

A revolute joint removes all but a single rotational degree of freedom from two objects.
The axis along which the two bodies may rotate is specified with a point and a direction
vector.
Prismatic joint

A prismatic joint permits relative translational movement between two bodies along an
axis, but restricts relative rotational movement. It is usually necessary to add joint limits
to prevent the bodies from getting too far from each other along the joint axis.
Distance joint

The distance joint maintains a certain minimum or maximum distance (or both) between
two points attached to a pair of actors. It can be set to be springy in order to behave like a
rubber band.
3. CRANE MODELLING METHOD
We divided the crane model into three parts as shown in Fig. 1: the suspension module, the lifted-object module,
and the manipulation module. The suspension module and the lifted-object module are the most important parts
of the crane model as they can directly influence the simulated physical behaviours. The suspension module
represents the rigging system of the cranes, including the trolley, cable, block, and hook. The lifted-object
module includes hooks, a lifting object, and optional lifting devices. In general, we can classify the lifted-object
module into two different types: lifting with a lifting device and without any lifting device, as shown in Fig. 2
(a) and (b) respectively. The manipulation module includes the crane body, wheels, cabin, and booms etc,
according to the type of cranes we used. In this paper, we focus on modelling a general type of mobile crane to
simulate cooperative crane operations.
238
9
th

FIG. 1: The cranes are divided into three parts: the suspension module, the lifted-object module, and the
manipulation module.

FIG. 2: The suspension module and the lifted-object module: (a) lifted-object module with a lifting device; (b)
lifted-object module without any lifting device.
3.1 Suspension module
The suspension module has four actors as shown in Fig. 3: the trolley actor, the cable actor, the block actor, and
the hook actor. The trolley actor is on the top of this module and is the adapter to the manipulation module.
Although the rigging system of the mobile crane doesnt have a trolley element, we retain this actor in the
suspension module for generalization purposes. We connect the trolley to the boom using a fixed joint in a
mobile crane; however, a prismatic joint is used in tower cranes.
A spherical joint is used to connect the trolley actor with the cable actor to simulate the swing effect of the
suspension system. In general however, we usually use more than one cable to support the block system in
cranes to lift heavy objects. When the block is twisted during the erection process, the multiple cables will force
the block to retain its original rotation along the system axis. To approach the physics behaviour described, we
apply a joint spring to the orientation along the system axis. However, according to Newtons second law, a
spring system will form a linear simple harmonic oscillator (David H. et al., 2005). In a real case however,
twisting motions of the block will weaken with time due to the energy lost in twisting the cables. To simulate
the energy loss, we apply a joint damper along the system axis in the spherical joint.
239
9
th
The connection between the cable and the block is a prismatic joint. The prismatic joint simulates the extension
and the retraction movements of hoisting operations by changing the length of the linear limit during erection.
The hook is attached into a block using a mechanism of 1-DOF in a general crane block. We use a revolute joint
to simulate the mechanism by attaching the block actor and the hook actor. During the erection process, the
lifting object will pull the hook down and increase the static friction between the hook and the block. Due to this
friction, this revolute joint will not be totally free to rotate. Therefore we apply a joint damper into the revolute
joint here to prevent the endless rotating motion and to simulate the realistic physical behaviours of hooks.
Finally, the hook actor will be attached to lifting objects to complete the suspension module.

FIG. 3: Suspension module: (a) Illustration of the suspension module; (b) Physics model of the suspension
module. The trolley actor will be fixed on the boom actor with a fixed joint in mobile cranes, and will be
connected to the boom actor with the prismatic joint in tower crane.
3.2 Lifted-object module
The lifted-object module includes the object to be erected and any optional lifting devices. The lifting device
(which is normally referred to as a spreader) is commonly used for transferring an evenly distributed load to the
crane (Cartmell et al., 1998). Several erection examples without a lifting device and with a lifting device are
shown in Fig. 4 and Fig. 5, respectively. The slings are the linkages between the hook, the lifting devices, and
the lifting objects. We simulate a sling using a distance joint. In Fig. 4, the lifted-object module without lifting
devices can be simulated as two actors directly connected with multiple distance joints. Lifting with lifting
devices can be modelled as an extension of lifting without lifting devices by adding a lifting device actor
between the hook and the object as shown in Fig. 5. The two actors, the hook and object, are indirectly
connected by a lifting device actor with multiple distance joints (slings). By combining the distance joints with
different relative attachment positions, we can realistically simulate the swinging behaviour of lifting and create.
240
9
th

FIG. 4: Lifted-object module without lifting devices: (a), (c), and (e) are illustrations of a lifting example with
different slings distributions; (b), (d), and (f) are physics models relative to (a), (c), and (e.)

FIG. 5: Lifted-object module with lifting devices: (a), (c), and (e) are illustrations of a lifting example with
different types of lifting devices; (b), (d), and (f) are the corresponding physics models.
241
9
th
3.3 Manipulation module
The manipulation module represents the main body of the crane. This model would be different for different
types of cranes. In previous works, the manipulation module was built using the closed-form forward kinematics
method (Chi et al., 2007). In order to simulate the detailed physical behaviours of crane bodies during the
erection process, we included multibody dynamics. By attaching joints on each actor of cranes, we can simulate
the real mobile behaviours of mobile cranes. Fig. 6 shows a general mobile crane model representation and its
physics model using multibody dynamics is shown in Fig. 7.

FIG. 6: An illustration of a general mobile crane model.

FIG. 7: The manipulation module of a general mobile crane.
4. IMPLEMENTATION
A computer system was developed to implement the proposed model of this paper. By changing the joint limit
and applying joint motors in the virtual environment, cranes can be operated using predefined scripts or by a
user using user interfaces such as a keyboard or a mouse. The developed system was implemented on the latest
graphics engine, Microsoft XNA, a rendering framework which is generally used to develop computer games
and Xbox360 games (Microsoft, 2008). PhysX, a well-known physics engine with a multibody dynamics solver
which uses the position based dynamics approach (Mller, 2007, NVIDIA, 2008), was also integrated in the
system.
242
9
th

FIG. 8: The cooperative erection project of the Dalin Oil Refinery.
To demonstrate the developed physics model, a cooperative erection project of the Dalin Oil Refinery in Taiwan
was simulated. The feasibility of the physics model shown in Fig. 8 was investigated. The lifting object of the
project, a vessel, weighed about 45 tons. It was initially lying down and needed to be erected and lifted to the
designated location. A main lifting mobile crane with a spread bar and a tail lifting mobile crane without a
lifting device were planned for the erection. To simulate this scenario, we built two mobile cranes (one
manipulation module and one suspension module each) and one lifted-object module as shown in Fig. 9. The
simulation result is shown in Fig. 10. From time 0 to time 100, the frames show an erection process using
cooperating cranes. Then, the main lifting crane lifted the object on its own to the designated location by
rotating the booms during time 100 to time 200. A smooth and realistic simulation was performed at 40 frames
per second running on a system equipped with an Intel Core2 Duo 2.13GHz processor with 3GB of RAM and a
Geforce 7950 display card.

FIG. 9: The model of the example case (not including the manipulation module of each crane): (a) illustration
of the cooperating cranes; (b) physics model of the cooperating cranes.
243
9
th

FIG. 10: The simulation process of the cooperative erection project.
5. CONTRIBUTIONS
This paper presented a physics-based crane model for simulating erection activities using cooperating cranes.
The major contributions are summarized as follows:
Configurability: The proposed method separated a rigging system into one or more manipulation
modules, lifted-object modules, and suspension modules. These modules are flexible and can be
reassembled to simulate different combinations of cranes. The configurability allows users to
simulate different types of cooperative erections.
Versatility: The proposed method can not only be applied to cooperative erections but can also be
applied to single crane operations. It can also be used to simulate tower cranes and the rigging
system used on construction sites.
Portability: The proposed model can be integrated with existing physics engines which support the
joint constraints. Some physics engines, such as PhysX, can even provide hardware acceleration.
This allows future developers to adopt it in other platforms.
High-fidelity and continuous simulation: The proposed method allows real-time simulations and
visualizations. The physics feedback is computed in real-time based on the users inputs. The
physics model can change dynamically. This allows the simulation of the entire erection cycle, in
which lifting devices and objects are dynamically attached and detached.
6. CONCLUSIONS
This paper presented a physics-based modelling method for cooperative cranes. We separated a crane into three
parts, the crane suspension module, lifted-object module, and the manipulation module, to develop a general and
complete crane model. A system was developed to demonstrate and test the feasibility of the proposed model
using an example case of a dual-crane scenario. The system performed a realistic and interactive simulation in
real time. The proposed crane model is extendable, enabling us to model different kinds of cranes and lifting
244
9
th
devices. Future work will focus on model verification to increase the reliability of crane models and improve the
proposed crane model.
7. REFERENCES
Amatucci, E., Bostelman, R., Dagalakis, N., and Tsai, T. (1997). Summary of modeling and simulation for
NIST RoboCrane. Proceedings of the 1997 Deneb International Simulation Conference and Technology
Showcase. Detroit.
Bernold, L., Lorenc, S., and Luces, E. (1997). On-line Assistance for Crane Operators. Journal of Computing in
Civil Engineering, 11 (4), 248-59.
Cartmell, M. P., Morrish, L., and Taylor, A. J. (1998). Dynamics of spreader motion in a gantry crane.
Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering
Science, 212 (2), 85-105.
Cheng, Z., and Amin, H. (2007). Collaborative Agent-based System for Multiple Crane Operation. Internetional
Symposium on Automation & Robotics in Construction.
Chi, H. L. (2007a). Physics-Based Simulation of Detailed Erection Activities of Construction Cranes. Master
Thesis.
Chi, H. L., Hung, W. H., and Kang, S. C. (2007). A Physics Based Simulation for Crane Manipulation and
Cooperation. Computing in Civil Engineering Conference. Pittsburgh, U.S.
CM Labs. (2007). Vortex Simulators. Retrieved 6 30, 2009, from Vortex:
http://www.vxsim.com/en/simulators/index.php
David, H., Robert, R., and Jearl, W. (2005). Foundamentals of Physics. Wiley.
Erleben, k., Sporring, J., Henriksen, K., and Dohlmann, H. (2005). Physics Based Animation. Hardcover.
GlobalSim. (2007). GlobalSim. Retrieved 06 30, 2009, from http://www.globalsim.com/
Huang, J. Y., and Gau, C. Y. (2003). Modelling and Designing a Low-cost High-delity Mobile Crane
Simulator. International Journal of Human-Computer Studies, 58 (2), 151-176.
Kamat, V. R., and Martinez, J. C. (2005). Dynamic 3D Visualization of Articulated Construction Equipment,
Journal of Computing in Civil Engineering, 19 (4).
Kang, S. C., and Miranda, E. (2008). Computational Methods for Coordinating Multiple Construction Cranes.
Journal of Computing in Civil Engineering, 22 (4), 252-263.
Kang, S. C., and Miranda, E. (2009). Numerical Methods to Simulate and Visualize Detailed Crane Activities.
Computer-aided Civil and Infrastructure Engineering, pp. 169-185.
Lipman, R., and Reed, K. (2000). Using VRML in Construction Industry Applications. Proceedings of
Web3D:VRML 2000 Symposium. Monterey.
Liu, L. (1995). Construction crane operation simulation. Proceedings of second Congress in Computing in Civil
Engineering. Atlanta.
Ali, A. D. M. S., Babu, N. R., and Varghese, K (2005). Collision Free Path Planning of Cooperative Crane
Manipulators Using Genetic Algorithm. Journal of Computing in Civil Engineering, 9 (2), 182-193.
Microsoft. (2008). XNA. Retrieved 7 14, 2009, from MSDN: http://msdn.microsoft.com/en-us/xna/default.aspx
NVIDIA. (2008). NVIDIA PhysX SDK 2.8 - Introduction. Santa Clara, U.S.A.
O'Connor, J., Dhawadkar, P., Varghese, K., and Gatton, T. (1994). Graphical visualization for planning heavy.
Proceedings of theFirst Congress on Computing in Civil Engineering. Washington DC.
Simlog. (2007). Mobile Crane Personal Simulators. Retrieved 6 30, 2009, from
http://www.simlog.com/personal-crane.html
Sivakumar, P., Varghese, K., and Ramesh Babu, N. (2003). Automated Path Planning of Cooperative Crane
Lifts Using Heuristic Search. Journel of Computing in Civil Engineering, 17 (3), 197-207.
245
9
th
Stone, W., Reed, K., Chang, P., Pfeffer, L., and Jacoff, A. (1999). NIST Research Toward Construction Site
Integration and Automation. Journal of Aerospace Engineering.
246
9
th

INTERACTION BETWEEN SPATIAL AND STRUCTURAL BUILDING
DESIGN: A FINITE ELEMENT BASED PROGRAM FOR THE ANALYSIS
OF KINEMATICALLY INDETERMINABLE STRUCTURAL
TOPOLOGIES
Herm Hofmeyer, Dr.
Associate Professor in Applied Mechanics, Eindhoven University of Technology, Department of Architecture,
Building and Planning, The Netherlands;
h.hofmeyer@tue.nl
Peter Russell, Univ.-Prof. Dipl.-Ing. M.Arch
Professor in Computer Aided Architectural Design (CAAD, Lehr- und Forschungsgebiet Computergesttztes
Planen in der Architektur), RWTH Aachen University, Germany;
russell@caad.arch.rwth-aachen.de
ABSTRACT: To understand the spatial and structural building design process and to help designers involved, the
idea of a research engine has been developed: In this engine cyclic transformations take place between spatial and
structural building designs. With this engine, a design process can be studied closely and subjected to improvement,
and designers can be supported. The transformation from spatial to structural design consists of four sub
transformations: (1) from spatial design to structural topology, (2) from structural topology to mechanical model,
(3) from mechanical model to finite element model, and (4) from finite element model to design recommendations.
Because in step (1) architectural elements are transformed into structural elements only, the resulting structural
topologies are not a-priori kinematically determined. Therefore, in step (2), one of the problems to solve is the exact
description of a kinematically indeterminable topology (at which nodes does spatial freedom exists and in which
direction). In this paper a method for this description and its implementation will be presented, developed at RWTH
Aachen University during a stay of the first author. The method starts with assigning specific finite elements, which
define a relationship between element nodal forces and displacements, to each architectural element: Architectural
columns are made by truss-elements, architectural beams with beam-elements, and architectural walls and floors
are transformed into flat shell elements. For each element the local stiffness matrix and local coordinate system are
calculated, whereafter global element stiffness matrices can be derived. These are merged into a system stiffness
matrix, which is subsequently reduced for boundary conditions and reduced elements. Now, the null space of the
stiffness matrix can be calculated, using the technique of singular value decomposition, which yields, after some
further processing, exactly defined modes of spatial freedom related to the structural topology. The method was
implemented for general 3D-cases in C++ and was checked for specific problems, which will also be presented in
the paper. For further transformations in the research engine, the modes of spatial freedom should be suppressed,
thus resulting in a kinematically determined structure that can be subjected to structural (and later architectural)
optimization. In this paper also some suggestions will be made for methods (to be developed), which will make the
design kinematically determined. The methods presented, partly implemented and partly in development, will be part
of the research engine and as such will provide support to spatial and structural building designers and will provide
insight and new developments in the design process of buildings itself.
KEYWORDS: Structural building design, spatial building design, generative design, finite element method,
kinematically indeterminable.
1. INTRODUCTION
Recently, the idea of a research engine has been developed, in which spatial building designs are modified or
transformed into structural building designs and vice versa by means of a cyclic procedure (Hofmeyer, 2007), Fig. 1
on the left. The research engine provides support in two domains. First of all, the transformation or modification
methods ("Trans. selection" in the right part Fig. 1) can be varied and the resulting spatial and structural design
evolution can be followed (by means of the "Measure" in the right part of Fig. 1), resulting in a study of the design
process. Secondly, the actual spatial and structural designs provide the designer with possible new solutions,
resulting in a study of generative design. The concept of the research engine is a general one and can be interpreted
247
9
th

in several ways, e.g. as a conventional architectural or structural design technique, as structural optimization, or as
multi-disciplinary optimization (Hofmeyer & Kerstens 2008). Within the research engine, the transformation from a
spatial design 2n-1 to a second structural design 2n consists of four sub transformations: (1) from spatial design to
structural topology, (2) from structural topology to mechanical model, (3) from mechanical model to finite element
model, and (4) from finite element model to design recommendations (Hofmeyer & Bakker 2008) and Fig. 1 on the
right. Note that sub transformation 3 is a single transformation (although two arrows are used in the figure) and
Struct. design 2n-1 is represented both by the preceding mechanical model and the finite element model that follows.
The first sub-transformation is a set of spatial-structural transformation rules that adds geometrical components
(lines and areas, which later will become structural elements like columns, beams, and plates) to a spatial design. It
is important to realise that a set of such geometrical components (a structural topology which represents a future
structural design) is likely to be kinematically undetermined, which means that components may not be connected at
all or only connected such that the topology is not stable: This means only a small imperfection will be enough for
the structural topology to collapse. For some technical reasons, to be discussed, this set of components therefore
cannot be used with finite element model 2 (see Fig. 1 at the right) to analyse stresses and to generate design
recommendations. It should first be changed in a so-called "mechanical model", a kinematically determined (stable)
model with the necessary boundary conditions (i.e. a foundation and loads). This problem cannot be solved easily.
For example a part of it, specifying the exact degrees of freedom as related to the kinematically undetermined state,
is already very difficult as was also noticed by others (e.g. Rottke 1998).

FIG. 1: Research engine on the left and right (grey), sub-transformations on the right (black).
Recently, some ideas on two methods to generate a mechanical model out of a structural topology (sub-
transformation 2) have been presented (Hofmeyer & Bakker 2008). In this paper, a realized part of one of these
methods is presented in-depth: A method to analyze the set of geometrical components in order to know which
degrees of freedom exist and where these degrees of freedom are located within the set of geometrical entities. The
analyzing method uses principles of the finite element method in sub-transformation 2 (shown as finite element
model 1), and should not be mistaken for the finite element model 2 that is used for stress analyses between sub-
transformation 3 and 4, see Fig. 1 on the right. Note that before the finite element models can be applied,
additionally the geometrical components should be redefined such that a conformal partitioned geometry occurs
(Hofmeyer & Gelbal 2009).
2. STATE-OF-THE-ART
Considering the research engine, in the field of Architecture, Engineering, and Construction many research projects
have been carried out to investigate the multi-disciplinary character of the field and to develop computer aided tools
248
9
th

to support the design processes involved (e.g. Fenves et al. 1994). In this paper, within the multi-disciplinary design
process, only the disciplines of spatial design and structural design are part of the problem definition. Related
research can be divided in three groups. The first group is descriptive research that develops data models, which
formalize data and their relationships regarding specific aspects of the design process. Related to this paper are data
models that have been specifically developed to relate spatial and structural design (Khemlani et al., 1998, Matthews
et al., 1998, Eastman and Jeng, 1999, Rivard and Fenves, 2000, Scherer and Ghere, 2000, Mora et al., 2006). The
second group is generative research that yields programs, procedures, or concepts for generating spatial and/or
structural design solutions. The oldest but still active field in this group is that of space-allocation that transforms
building requirements into a spatial design (e.g. Kotspoulos 2006). For structural design, a distinction should be
made between research that optimises an existing structural design by means of expert systems, form-finding or
optimization (e.g. Rafiq et al., 2003, Bletzinger and Ramm, 2001, Kocaturk et al. 2003) and research that results in
the actual one-way transformation and evaluation from a spatial to a structural design (Rafiq & MacLeod, 1998,
Fenves et al., 2000). For most research in these two groups, the basic underlying idea is that in the design process a
more or less one-way path runs from spatial to structural design. However, the building design process can also be
modelled with a more cyclic approach, as shown in Fig. 1 on the right (grey). A start is made in cycle n by the
transformation of spatial design 2n-1 into structural design 2n-1, which is often carried out by a structural engineer.
The resulting structural design 2n-1 will be subject to improvement, for example by expert views of other structural
engineers or by optimization techniques. This optimised structural design 2n will be given to the architect and he
will then adjust the spatial design 2n-1 to fit the structural design, which gives spatial design 2n, or to fulfil other
requirements from the building plan yielding spatial design 2n-1 for the next cycle (n increases by 1). The resulting
design cycle -as shown in Fig. 1 on the right (grey)- is defined as "interaction between spatial and structural design"
and the use of this model of the design process in the research engine is justified by many research projects in the
third group, namely on the support of multi-disciplinary design processes, e.g. a building design project can be seen
as a sequence of views and dependencies from several disciplines (Haymaker et al., 2004).
Specifically concerning the problem as defined in this paper, the analysis of kinematically indeterminable structural
topologies, only a very limited amount of information can be found in literature. This is mainly because the use of
the finite element method normally presumes a kinematically determined structure and engineers are educated to
verify whether the structure is kinematically determined before the use of the method. And if their check would not
be rigorously enough, they normally know how to interpret the resulting error signals of the method (extreme
displacements, singular matrix warnings, etc.). Also, a building structure should always be kinematically determined
to be able to be properly used and in the preliminary design phase this is accounted for explicitly. Once the building
structure is then verified by means of the finite element method, kinematically indeterminable structures are not an
issue. What is behind these two explanations is that making a structure (and thus a structural topology) kinematically
determined is a design problem: in general several solutions exist for this and only the structural engineer's
experience and preferences will determine the final design. And because it is a design type problem, it is difficult to
find general applicable methods for support and assistance. Nevertheless, at least for one finite element program (a
user-friendly type and meant for the practical structural engineer) it is known that is helps the structural engineer,
once a kinematically indeterminable structure has been used as input, by suggesting additional structural supports
(MatrixCAE, 2009). This is, however, a limited approach as also more elements (like beams or plates) may be
needed. Finite element programs for more general and academic use (e.g. Ansys Inc., 2007) do not provide such
helpful information. In the field of dynamics, free body motions, which also occur for structures that are defined as
kinematically undetermined in this paper, need not to be a problem if dynamic behaviour is analysed. In several
textbooks this subject is elaborated, including the notion of a matrix's null-space to find the number of free body
modes (e.g. De Kraker and Van Campen, 2001). It can be concluded that procedures for (1) automated analyses of
kinematically indeterminable structural topologies and for (2) automated suggested solutions for a kinematically
determined structural topology are not available in literature and in existing software. Existing literature suggests the
matrix's null-space as helpful in the analysis of kinematically indeterminable structural topologies.
In this paper a matrix's null-space will be used to solve problem (1), the automated analyses of kinematically
indeterminable structural topologies. First, in section 3 the elements (truss, beam, and flat shell) used for the
analysis will be presented, followed in section 4 by a brief introduction of null-space and the algorithms needed to
find it. Then in section 5 a C++ program is presented, as procedure within the partly implemented research engine,
which is used to process some examples shown in section 6. Finally, section 7 and 8 present future methods and
conclusions respectively.
249
9
th

3. ELEMENTS
As discussed in section 1 and 2, the finite element method will be a part of the method to analyse kinematically
undeterminable building structures. In this paper no extended review of the finite element method will be given (e.g.
Zienkiewicz and Taylor, 1998) but for the sake of understanding its basic concept will be presented here. In general,
finding displacements, strains and stresses for loaded structures is not possible due to the fact that the displacement
field in the continuous structure is not known exactly. Therefore, the structure is split up in small parts with nodes,
so called finite elements, for which the displacement field within the elements is approximated such that the
relationship between nodal displacements and nodal loads can be determined, for instance by using the principle of
stationary potential energy. Setting equal the displacement of coincident nodes for all elements, some administrative
procedures then yield the system stiffness matrix that relates external loads and nodal displacements. Finally, an
equilibrium equation containing this system stiffness matrix is solved, leading to approximated strains and stresses
for every location in the structure. For the method presented in this paper, three finite element types have been
selected: A beam element, a truss element (derived from the beam element) and a flat shell element.
3.1 Beam element
For the analysing method, a beam element has been selected as presented by Przemieniecki (1968) and shown in
Fig. 2 on the left.

FIG. 2: On the left a beam element as shown in Przemieniecki (1968), on the right a flat shell bending element
developed by Batoz and Tahar (1982).
It is straight and has a uniform cross-section. Two nodes, i and j, and six degrees of freedom (DOF) according to the
local coordinate system, exist for each node. Consequently, each node can resist axial forces, a twisting moment,
two shear forces, and two bending moments. Engineering beam theory can be used to derive the element stiffness
matrix (the relationship between nodal forces and displacements), which means that, given the limitations of beam
theory, the element is not an approximation but yields correct results for the displacements and strains/stresses. The
local element x-axis is defined from node i to node j and the local y-axis is perpendicular to it and in the global x-y-
plane. For the special case that the local x-axis is parallel to the global z-axis, the local y-axis is arbitrarily set equal
to the global y-axis. Finally, the local z-axis is defined by making a right-handed Cartesian local coordinate system.
The beam element has 6 zero energy modes (3 translations and 3 rotations in space) and no spurious modes.
3.2 Truss element
A truss element can only resist axial forces and no shear forces, bending moments, or twisting moments. It can be
derived from the beam element presented in the previous section by removing all terms in the element stiffness
matrix related to the latter mentioned forces and moments. Using Fig. 2 on the left, this means removing possible
250
9
th

stiffness terms in the stiffness matrix for all rows and columns except 1 and 7. The truss element has 5 zero energy
modes (3 translations and only 2 rotations in space as rotation along the beam axis is not supported by the node
DOF's) and no spurious modes.
3.3 Flat shell element
For modelling floors and walls, a flat shell element can be used as shown in figure 2 on the right. It is assumed to be
flat and its node numbers should be defined (counter) clockwise. The local coordinate system is defined by the
origin as the intersection of diagonals, the x-axis from the origin to the middle of the line between node j and k, the
y-axis in plane and perpendicular to the x-axis, and finally the z-axis is defined such that a right handed Cartesian
system occurs. Its plane stress behaviour is based on a presentation of such elements in Cook et al. (1989) and its
bending behaviour is based on a so-called DKQ element as developed by Batoz and Tahar (1982). After the authors
repeated the derivations in the last mentioned paper, some typing errors were found in the equations, which are
available via the first author. For both cases (plane stress and bending) numerical integration is applied, 2 * 2 over
the surface. For the drilling stiffness (the ratio between applied force and displacement S
6
, S
12
, S
18
, and S
24
in Fig. 2
on the right) 1/1000 of the maximal element stiffness matrix diagonal value is used, as suggested by Kansara (2004).
As a result, the element has 3 zero energy modes regarding its bending part (1 translation and 2 rotations) en 3
modes regarding the in-plane behaviour (2 translations and 1 rotation) and no spurious zero energy modes.
4. NULL SPACE
Once a structural topology (a set of lines and areas) has been translated into a set of beam, truss, and flat shell
elements, for each finite element in this set the local coordinate system and the element stiffness matrix conform this
local coordinate system can be calculated. Then each local element stiffness matrix should be translated into an
element stiffness matrix conform the global coordinate system. This can be carried out using transformation rules as
presented in e.g. Cook (1989). By assuming that for each global node, consisting of all coincident element nodes,
equilibrium exists, and assuming that for all coincident element nodes the same displacements are applicable, the
(system) stiffness matrix for the complete structural topology can be composed. Finally, the system stiffness matrix
is reduced by processing the boundary conditions. Fig. 3 shows a simple structure of two elements and its system
stiffness matrix in global coordinates.

FIG. 3: System stiffness matrix of a simple structure having two truss elements.
The system stiffness matrix relates the forces/moments F
Si
and displacements/rotations d
Si
for each degree of
freedom as shown in the figure. Note that for node 1 no degrees of freedom are available because all displacements
are zero (simple support). For node 2, all three degrees of freedom exist, and for node 3, due to two roller supports,
only a displacement in x-direction is possible. If the structure stiffness matrix is singular, i.e. the inverse of the
matrix cannot be determined, the structure is not kinematically determined, in other words: one or more mechanisms
in the structure exist. However, this information is not enough and a method is needed to pinpoint the locations of
the mechanisms in the topology. This method may be based on finding the null space of the structure stiffness
matrix, for which the null space is indicated by a specific set of vectors that satisfy the following condition:
(1)
in which K is the stiffness matrix, and n is a vector of the matrix null space (Kreyszig, 1983). Vectors of the null
space may be summed and may be multiplied with a constant and in all these cases the new vector is part of the null
space of the matrix. Thus null space itself is unique, but its vectors are not. The null space of the matrix of this
system of equations can be determined by converting the matrix in row echelon form U, defining free variables in a
251
9
th

new vector x, setting them equal to zero except for one, and solving Ux=0 for each free variable (Hoffman and
Kunze, 1971). Note that it does not matter whether forces do work or not on the structure. Working forces would
only mean that the left term of equilibrium equation in Fig. 3 is non zero, but for the null space only a vector at the
right side that yields the vector at the left side zero is searched for. Vectors of the null space provide information on
which part of the structure forms a mechanism and/or whether the whole structure is able to transform or rotate in
space. This is shown by two vectors that form the null space of the structure in Fig. 3:
(2)
The first vector indicates a free movement of node 2 in y-direction (by a value for S5, the second row in the equation
of Fig. 3) and the second vector shows a free movement of node 2 in z-direction (S6). It should be noted that due to
(large) free movements of node 2 in y and/or z-direction, also node 3 will be able to move (less severe) free in x-
direction. However, this is a second-order effect and thus is not included in the null-space of a (linear) system of
equations. This example suggests the following:
(1) Every kinematically mechanism has a vector in the null space. This can be proved because a mechanism can be
described by a vector, and this vector is part of null space because no forces are needed for the mechanism thus
making the vector at the left side of the equilibrium equation in Fig. 3 equal to zero.
(2) If two mechanisms are connected, the two vectors of null space are still independent, i.e. they do not contain
displacement info for the same nodes. The proof development should take into account the derivation of the
structure stiffness matrix and possibly the algorithm with which the null space is found. As will be shown in section
6, although indeed the number of vectors corresponds to the number of mechanisms, the vectors itself are not
necessarily independent.
5. C++ PROGRAM
It is very laborious to find the structure stiffness matrix and its null space for realistic building related structural
topologies by hand. Therefore a program has been developed to automate this work, also because by using a
program the analysis of kinematically indeterminable structural toplogies can be more easily integrated in the
research engine, which was presented in section 1 and Fig. 1. The program has been written using the programming
language C++ via the Eclipse Platform version 3.4.1, Cygwin version 1.90.4.1, and Microsoft's Windows XP. It uses
input of the geometry of trusses, beams, and flat shells, whereafter for each element the local coordinate system, the
local, and the global element stiffness matrix are derived. The structure stiffness matrix is then composed out of the
element matrices and boundary conditions are incorporated. The matrix's null space is found via singular value
decomposition (Golub and Kahan, 1965), making use of the GNU Scientific Library GSL (GSL Team, 2007).
Finally, the program interprets the null space and outputs the degrees of freedom associated with each null vector,
for example as shown in section 6.1 for truss and beam elements.
6. DEMONSTRATION EXAMPLES
In this section some examples of structural topologies will be presented. They can be used to verify the elements and
program presented and to provide information needed to develop kinematically determined structural topologies.
The latter will be presented in section 7.
6.1 Element verification
The null space of a stiffness matrix not only indicates mechanisms in a structural topology but, because it defines the
singularity of the stiffness matrix, it also shows element formulation problems such as spurious zero energy modes,
which can be interpreted as element deformations not related to nodal forces. In this section 6.1, the elements as
presented in section 3 will be used as input for the program using no boundary conditions and the results can be used
to verify the elements' normal zero energy modes (rigid body movements) and if relevant, spurious zero energy
modes. Furthermore, the output provides information on how to interpret output for more complex cases in the next
sections. For a single truss element (shown in figure 2 on the left, but now without rotational degrees of freedom),
252
9
th

positioned parallel to the x-axis, and without boundary conditions, 5 vectors of the matrix null space occur with the
following free mechanisms/movements as output:
Free DOF (x) at finite element node 1 (keypoints 1), vector 1, component value: 0.707107
Free DOF (x) at finite element node 2 (keypoints 2), vector 1, component value: 0.707107

Free DOF (z) at finite element node 1 (keypoints 1), vector 2, component value: 1

Free DOF (y) at finite element node 1 (keypoints 1), vector 3, component value: -1

Free DOF (y) at finite element node 2 (keypoints 2), vector 4, component value: 1

Free DOF (z) at finite element node 2 (keypoints 2), vector 5, component value: 1
The first vector indicates free movement of node 1 and node 2 in x-direction, which is correct because in x-direction
node 1 and 2 are connected by the truss. All other vectors indicate movement of an end node in y or z-direction.
Because the element is positioned parallel to the x-axis, these vectors present the most fundamental possibilities for
mechanisms and, as discussed in section 4, combinations of these vectors can be made for more complex free-body
movements. If the element is positioned arbitrarily (not parallel to the Cartesian axes), vectors result that are a
combination of x, y, and z-displacements and consequently these vectors cannot be linked with the rigid body modes
intuitively. No spurious zero-energy modes exist.

For a single beam element, as shown in figure 2 on the left, without boundary conditions, 6 vectors of null space are
listed in the output. Vector 1 to 3 all describe (unequal) translation in the y-direction for both nodes combined with
corresponding (equal) rotations around the z-axis and a free (equal) rotations around the x-axis. Also a very small
translation in the z-direction exists, for which the corresponding rotations (ry) disappear in numerical noise. Vector
4 and 5 represent a transformation in the z-direction for node 1 and 2 respectively with corresponding rotations. Note
that these rotations in radians are very small compared to the translations and as all computer output has some
numerical noise it is important to distinguish correctly between an actual degree of freedom or noise. Finally, vector
6 shows a translation in the x-direction and is equivalent to the truss example vector 1. Six vectors exist,
corresponding to six possible rigid body displacements (x, y, and z-translations and rx, ry, and rz-rotations), but the
output shows that even for elements aligned with the global coordinate system, vectors do not present the most
fundamental possibilities for mechanisms. It seems not to be easy to generate these most fundamental possibilities,
at least making the vector orthogonal is not enough (De Kraker and Van Campen, 2001, Fig. 3.5 and 3.6). No
spurious zero-energy modes exist.
For the flat-shell element a similar situation as for the beam element exists. The output consists of 6 vectors,
corresponding to the six possible rigid body displacements, but not presenting the most fundamental possibilities for
the rigid body modes displacements (x, y, and z-translations and rx, ry, and rz-rotations). No spurious zero-energy
modes exist. However, if the flat shell has a very small thickness, for certain boundary conditions its low bending
stiffness may occur as a spurious zero-energy mode. Now very low values of the diagonal matrix (associated with
the singular value decomposition) should be neglected. It is thus clear that, due to numerical noise, the theoretical
difference between a pure kinematical mechanism and a very low stiffness may be difficult to find.
6.2 Structural topologies
To investigate the output for structural topologies being a concept of real building designs, in this section a specific
structural topology is designed as shown in Fig. 4. It has two levels, to investigate whether a difference in output
exists for two independently unstable levels or the top level being unstable but relative to a stable first level.
Furthermore the elements drawn with dotted lines can be added or left out to investigate the output for rotational
stability, as shown in the table in Fig. 4.
The first model A (72 DOF's), consisting of a stable first level and a complete unstable second level, yields 3 vectors
of the null-space. These vectors represent free displacements in x-direction and in y-direction, and a rotation around
the z-axis. Again, the vectors represent these free displacement modes not in their most fundamental form, but as
combinations of these forms. Also, the rotation around the z-axis is expressed in x and y-displacements of the nodes,
as the rotation around the z-axis of the flat-shell element used is not coupled to the in-plane x and y-displacements.
Model B, having only a rotational unstable second level, shows a single vector of null-space, indicating a rotation
around the z-axis of the upper flat-shell element.
253
9
th

Finally model C, totally unstable, yield 6 vectors of the null-space. These are linked to the three x, y, and rz
displacements/rotations of the first level and the three displacements/rotations of the second level. Unfortunably,
every vector contains displacements for all nodes and thus the vectors do not represent the free displacement modes
in their most fundamental form.

FIG. 4: Structural topology (72 DOF's) representing an actual building structure.
7. FUTURE RESEARCH
Now that it is possible for a kinematically indeterminable structural topology to analyse the mechanisms involved,
the next research question is how this structural topology can be made kinematically determined automatically.
Here, some very brief suggestions for the solution of this problem will be presented, which are to be researched in
the near future. One suggested method is to start with a node having free DOF's as close a possible to the ground
supports and being one of the most outer nodes of all other nodes on its height. If its free DOF-components are in x-
direction and there is another node fixed in x-direction at ground level and having the same y-coordinate then a truss
or shell element (the latter having a diagonal equal to the truss element) can be added between the two nodes. For
the y-direction a similar procedure can be carried out. If no additional nodes to the node investigated can be found,
the next node (most outer) on this level could be tried, using the same procedure. An alternative method is to use
also nodes in the "building", which are nodes being not the most outer. In this case they can be connected to nodes
in the same horizontal plane. After each addition of an element, the procedure as presented in section 3 to 6 should
be repeated to study the effect. Only if all nodes at a certain level are fixed in space, the next level can be subject to
the methods. Using one of these two methods (or a combination of both), the structure is made stable from the
ground (supports) up. A third method is to observing the null vectors and if two nodes having the same degree of
freedom (but with different values) and positioned in a (preferably outer) surface parallel to the degree of freedom,
they are coupled. This process is then also repeated until the system is completely determined. Note that using the
suggested methods it will not be uncommon to generate statically overdetermined structures. However, this is not a
problem for further steps like finite element processing.
8. CONCLUSIONS
As part of a so-called research engine, transforming spatial designs to structural designs and vice versa to research
design processes and relationships, a method has been presented to analyse kinematically undeterminable structural
254
9
th

topologies. The method has been implemented in a C++ program and is able to use finite truss, beam, and flat shell
elements and it determines the null space of the structure's stiffness matrix. The program have been used to check
that the elements programmed do not show spurious zero energy modes.
A typical building example shows that although the method functions correctly and the number of vectors
corresponds to the number of mechanisms, the vectors of the null-space do not present the mechanisms in the most
fundamental way (i.e. are not independent).
Due to numerical noise, the theoretical difference between a pure kinematical mechanism and a very low stiffness
may be difficult to find.
Given the output of the program, three methods to design a kinematically determined structural topology have been
briefly proposed. They will be researched in the near future.
Parallel to this research is the trend to develop BIM-models which have, in opposition to the past where only data
was included in the set-up, now a more structuring role (Russell and Elger, 2008). This research could be seen as a
related study to the latter role.
9. REFERENCES
Ansys Inc. (2007). Ansys 8.1, Ls-dyna 970 rev 3858, and user manuals, ANSYS, Inc., Southpointe, 275 Technology
Drive, Canonsburg, PA 15317, USA.
Batoz, J.-L. and Tahar, M.B. (1982). Evaluation of a new quadrilateral thin plate bending element, International
journal for numerical methods in engineering, Vol. 18, 1655-1677.
Bletzinger K.U. and Ramm E. (2001). Structural optimization and form finding of light weight structures,
Computers & Structures, Vol. 79, No. 22-25, 2053-2062.
Cook, R.D., Malkus, D.S. and Plesha, M.E. (1989). Concepts and applications of finite element analysis, third
edition, John Wiley & Sons, New York.
De Kraker, B. and Van Campen, D.H. (2001). Mechanical Vibrations, Shaker Publishing, Maastricht, The
Netherlands.
Eastman C. and Jeng T.S. (1999). A database supporting evolutionary product model development for design,
Automation in construction, Vol. 8, No. 3, 305-323.
Fenves S., Flemming U., Hendrickson C., Maher M.L., Quadrel R., Terk M. and Woodbury R. (1994). Concurrent
Computer-Integrated Building Design, PTR Prentice Hall, Englewood Cliffs, New Jersey.
Fenves S.J., Rivard H. and Gomez N. (2000). SEED-Config: a tool for conceptual structural design in a
collaborative building design environment, Artificial Intelligence in Engineering, Vol. 14, 233-247.
Golub, G.H. and Kahan, W. (1965). Calculating the singular values and pseudo-inverse of a matrix, Journal of the
Society for Industrial and Applied Mathematics, Series B, Vol. 2, No. 2, 205224.
GSL Team (2007). "13.4 Singular Value Decomposition", GNU Scientific Library, reference manual,
www.gnu.org/software/gsl/.
Haymaker J., Fischer M., Kunz J. and Suter B. (2004). Engineering test cases to motivate the formalization of an aec
project model as a directed acyclic graph of views and dependencies, ITcon, Vol. 9, 419-441.
Hoffman, K.M. and Kunze, R. (1971). Linear algebra (2nd edition), Prentice Hall.
Hofmeyer, H. (2007). Cyclic application of transformations using scales for spatially or structurally determined
design, Automation in Construction, Vol. 16, No. 1, 664-673.
Hofmeyer, H. and Bakker, M.C.M. (2008). Spatial to kinematically determined structural transformations, Advanced
Engineering Informatics, Vol. 22, No. 3, 393-409.
255
9
th

Hofmeyer, H. and Gelbal, F. (2009). Redefinition of geometrical components to specify kinematically undetermined
behaviour. Proceedings CIB-W78, 26
th
International Conference, Management IT in Construction, October
1-3, 2009, Instanbul, Turkey.
Hofmeyer, H. and Kerstens, J.G.M. (2008). A bird's-eye view of cyclic spatial-structural transformations, grammar
based design, and structural optimization, in. (Muylle, M. Ed.) eCAADe 2008, Architecture 'in computro',
Integrating methods and techniques, Proceedings of the 26th Conference on Education and Research in
Computer Aided Architectural Design in Europe, September 17-19, 2008, eCAADe and Artesis, Antwerp,
Belgium, 483-492.
Kansara, K. (2004). Development of membrane, plate and flat shell elements in Java, M.Sc.-thesis, Virginia
Polytechnic Institute and State University, Blacksburg, Virginia.
Khemlani L., Timerman A., Benne B. and Kalay Y.E. (1998). Intelligent representation for computer-aided building
design, Automation in Construction, Vol. 8, No. 1, 49-71.
Kocaturk, T., Van Weeren, C. and Kamerling, M.W. (2003). Integrating Architectural and Structural Form-Finding
Processes for Free-Form Digital Architectures, in (Bontempi, F. Ed.) ISEC-02, System-based Vision for
Strategic and Creative Design, Volume 1, Balkema, Lisse, The Netherlands, 557-562.
Kotsopoulos S.D. (2006). Constructing Design Concepts, A Computational Approach to the Synthesis of
Architectural Form, Massachusetts Institute of Technology (MIT), USA.
Kreyszig, E. (1983). Advanced Engineering Mathematics, Fifth Edition, New York, John Wiley & Sons Inc.
MatrixCAE (2009). MatrixFrame, Wijchenseweg 116, 6538 SX Nijmegen, The Netherlands, +31-24 34 34 380,
www.matrix-software.com/uk/
Matthews K., Duff S. and Corner D. (1998). A model for integrated spatial and structural design of buildings, in
(Sasada T., Yamaguchi S., Morozumi M., Kaga A. and Homma R. Eds.) CAADRIA '98: Proceedings of The
Third Conference on Computer Aided Architectural Design Research in Asia, Osaka University Japan,
Osaka, 123-132.
Mora R., Rivard H. and Bdard C. (2006). Computer representation to support conceptual structural design within a
building architectural context. Journal of Computing in Civil Engineering, Vol. 20, 76-87.
Przemieniecki, J. S. (1968). Theory of Matrix Structural Analysis, McGraw-Hill, New York.
Rafiq M.Y. and MacLeod I.A. (1998). Automatic structural component definition from a spatial geometry model,
Engineering Structures, Vol. 10, No. 1, 37-40.
Rafiq, M.Y., Mathews, J.D. and Bullock, G.N. (2003). Conceptual Building Design-Evolutionary Approach,
Journal of computing in civil engineering, Vol. 17, No. 3, 150-158.
Rivard H. and Fenves S.J. (2000). A representation for conceptual design of buildings, Journal of Computing in
Civil Engineering, Vol. 14, 151-159.
Rottke, E. (1998). ExTraCAD, Computeruntersttzung des architektonischen Traagwerkentwurfs, RWTH, Aachen,
Germany.
Russell, P. and Elger, D. (2008). The Meaning of BIM, Towards a Bionic Building, in. (Muylle, M. Ed.) eCAADe
2008, Architecture 'in computro', Integrating methods and techniques, Proceedings of the 26th Conference
on Education and Research in Computer Aided Architectural Design in Europe, September 17-19, 2008,
eCAADe and Artesis, Antwerp, Belgium, 531-536.
Scherer R.J. and Gehre A. (2000). An approach to a knowledge-based design assistant system for conceptual
structural system design, in (Goncalves, Stegier-Garcao, Scherer Eds.) Product and Process Modelling in
Building and Construction, Balkema, Rotterdam.
Zienkiewicz, O.C. and Taylor, R.L. (1998). The Finite Element Method, fourth edition, volume 1, basic formulation
and linear problems, McGraw-Hill book company, London.

256
9
th
InternationalConferenceonConstructionApplicationsofVirtualReality Nov56,2009
VIRTUAL ENVIRONMENT ON THE APPLE IPHONE/IPOD TOUCH

Jason S. Breland
School of Construction, The University of Southern Mississippi
Hattiesburg, MS 39402, USA
jason.s.breland@usm.edu
Mohd Fairuz Shiratuddin, Ph.D.
mohd.shiratuddin@usm.edu
ABSTRACT: 3D architecture and construction visualization applications are mostly deployed on desktop, laptop,
and tablet computing devices. Very few 3D visualization applications are developed for mobile devices due to
limitations in computing power and memory. Although there are portable handheld gaming devices that use some
form a 3D game engine to handle 3D graphics quite remarkably, they were never intended for serious 3D
application development.
Recently, mobile devices that can act as a phone or wireless device, a PDA as well as an entertainment system have
been introduced into the consumer market around the world, such as the Apple iPhone, and the Apple iPod Touch.
The rapid change in technology and its affordability has migrated 3D visualization application into these small
ubiquitous mobile devices. Their ability to connect to the Internet wirelessly through Wi-Fi, and/or Third
Generation (3G) network connections, allows for sharing and transferring of digital information among users
regardless of location, at variable speeds. These new mobile devices are also capable of executing 3D applications,
for entertainment and real-time visualization purposes.
Using the Unity game engine for the iPhone/iPod Touch, the authors developed a real-time 3D walkthrough for a
virtual environment application for iPod Touch. In this paper, the authors describe the development process, the two
unique user interaction techniques only available on the iPhone/iPod Touch, the benefits and the challenges, and
finally the potential future improvements to be developed for use in the construction industry.
KEYWORDS: Game engine, mobile, iPhone, iPod Touch, virtual environment
1. INTRODUCTION

Two decades ago, the Newton was developed and introduced by Apple Computers as the first personal digital
assistant (PDA) device. The Newton and other PDAs in the market were designed for personal use and were not
networked for communication purposes. However in recent years, advancements in the communication, Internet and
wireless technologies have led to the emanation of smart-mobile devices that combine features of PDAs, mobile
phones, and handheld game consoles. The iPhone/iPod Touch is one of the most recent smart-mobile devices
introduced by Apple Inc. One main advantage of the iPhone/iPod Touch and similar device is mobility, where users
can carry them anywhere. The ability to connect to wireless networks (3G or Wi-Fi) using the iPhone/iPod Touch
allows for ubiquitous voice and data communication.
The computing power of the iPhone/iPod Touch not only can handle voice and data communications, it can also
handle 2D and 3D handheld quality console games just like the ones running on the Sony Playstation Portable and
Nintendo DS/DSi. Having such computing power opens up new possibilities for the iPhone/iPod Touch to be used
as an architecture and construction 3D visualization tool for teaching and learning purposes as well as in the field.
Devices like the iPhone are now becoming viable tools for offline learning through storage of PDF files for later use
or review (Moore, 2009). More and more portable devices like the iPhone/iPod Touch are being assimilated into
work environments. This paper outlines the development process of a 3D real-time walkthrough in a virtual
environment (VE) application on the iPhone/iPod Touch using the Unity game engine. The authors also describe the
development process, the two unique user interaction techniques only available on the iPhone/iPod Touch, the
257
9
th
benefits and the challenges of development, and finally the potential future improvements that can be developed for
use in the construction industry.
2. IPHONE AND IPOD TOUCH

The iPhone and the iPod Touch were designed, developed, and marketed by Apple Inc. It is the first mobile device
that supports multi-touch and tilt-control interaction techniques. The iPhone and iPod Touch have similar
functionalities, except the latter does not function as a phone, and does not have built-in camera or microphone. In
Table 1, the authors summarize the comparisons between the iPhone and iPod Touch.
2.1 Specifications of the Iphone/Ipod Touch
TABLE 1: A comparison between the iPhone and iPod Touch (Apple, 2009a & 2009b)
iPhone iPod Touch
Dimensions &
Weight
Height: 4.5 inches (115.5 mm)
Width: 2.4 inches (62.1 mm)
Depth: 0.48 inch (12.3 mm)
Weight: 4.8 ounces (135 grams)
Height: 4.3 inches (110 mm)
Width: 2.4 inches (61.8 mm)
Depth: 0.33 inch (8.5 mm)
Weight: 4.05 ounces (115 grams)
Display 3.5-inch (diagonal) widescreen
Multi-Touch display with a 480-
by-320-pixel resolution at 163
pixels per inch
3.5-inch (diagonal) widescreen Multi-Touch
display with a 480-by-320-pixel resolution at
163 pixels per inch
Storage Capacity 16GB or 32GB flash drive
Holds up to 20 hours, or 40 hours
of video
8GB, 16GB, or 32GB flash drive
Holds up to 10 hours, 20 hours, or 40 hours of
video
Operating System iPhone OS iPhone OS
Battery life Internet use:
Up to 5 hours on 3G
Up to 9 hours on Wi-Fi
Video playback time of up to 10
hours when fully charged
Audio playback time of up to 30
hours when fully charged
Internet use:
Up to 9 hours on Wi-Fi

Music playback time of up to 36 hours when
fully charged
Video playback time of up to 6 hours when
fully charged
Camera, photos,
video recording
Yes Not Available
Microphone Yes Not Available
Network
Connectivity
UMTS/HSDPA (850, 1900, 2100
MHz), GSM/EDGE (850, 900,
1800, 1900 MHz), Wi-Fi
(802.11b/g), Bluetooth 2.1 + EDR

Wi-Fi (802.11b/g)

The iPhone operating system (OS) is relatively simple compared to the Mac OS X that runs on most Apple
computers. The iPhone OS takes up less than 240 Megabytes of the overall storage area of the device. This is
considered small compared to the complexity and functionality of the device. One disadvantage of the OS is its
inability to run multiple applications at once. For example, a user cannot run an instant messaging application in the
background while surfing the Internet. Another drawback is the battery life per full charge. Processing intensive
application such as 3D games, continuous data and voice communication to the wireless network, or playing videos,
reduce the battery life per full charge.
Apple recently releases the iPhone Software Development Kit (SDK) to encourage development of applications and
games from the end-users. This has encouraged end-users to create many innovative and useful applications and
258
9
th
games for the iPhone/iPod Touch. Applications and games that are approved by Apple are marketed and sold on the
Apples Apps Store to other users worldwide.
2.2 Limitations of the Iphone/Ipod Touch

According to industrial reviews, the iPhone/iPod Touch lacks some of the features which were expected such as
concerning the touch screen. The size of the touch screen is small that it becomes difficult to click on with great
accuracy. This perhaps becomes most apparent when the user first tries to use the default keyboard which appears
any time texts must be entered. Some users may find that entering little text such as a single sentence is now an
arduous task. In addition, the screen will become smeared with fingerprints. Other limitation include, the camera not
being able to zoom or record video. There is no simple battery replacement to extend battery life, or in case of
malfunction. Another limitation is the network. iPhone users must use the network that Apple has an agreement with
in the users country. If users do have the data plan, all internet resources are tethered to the device thus not
accessible by a laptop or other device (Block, 2008).
Another drawback the authors have found is the inability of the iPhone/iPod Touch to run more than one application
at a time. It can be assumed that Apple most likely opted to avoid application multitasking, or tasks running in the
background, as a way to preserve battery life and to avoid overtaxing the limited processing capabilities. With one
or more active program(s) running in the background, the battery could easily drained in a few hours. The exact
processing power of the iPhone and iPod Touch is absent from any known Apple posted information regarding the
devices. In fact users had to create applications specifically designed to tax and record resources just to find out the
exact processing power of the devices at runtime. Snell (2008) found:
The iPhone, iPhone 3G, and original iPod Touch all report a 400 MHz CPU and 100 MHz bus. The new
iPod Touch reports a 532 MHz CPU and 133 MHz bus. The physical memory remains the same across all
hardware, at about 117 MB (which Hockenberry interprets, I think correctly, as meaning 128 MB of RAM
with 11 MB being used for video). (Snell, 2008)
3. GAME ENGINES FOR THE IPHONE/ITOUCH

There are several game engines currently available for the iPhone/iPod Touch. Some are specific only for
developing 2D or 3D games, and some support both 2D and 3D games. When Apple first released the iPhone SDK
to the public, the only way custom applications or games can be developed was to use the Objective-C programming
language in the XCode development environment. Objective-C is the native language for the iPhone/iPod Touch
SDK. Developing an application or game using Objective-C can be overwhelming for new developers as all the
elements e.g. the Graphical User (GUI), the game-mechanics, physics, lighting, terrain, etc. of the game must be
created afresh.
However, with the release of the SDK and large interest from the development community, new game engines for
the iPhone/iPod Touch were developed for commercial use and others as freely distributed as Open Source. Table 2
shows some examples of the currently available game engines for development of 3D games on the iPhone/iPod
Touch. Unlike game engine for the PC, the number of 3D game engines available for the iPhone/iPod Touch is still
limited.
TABLE 2: Examples of 3D game engines for the iPhone/iPod Touch
Game Engine Developer Status Website Link
Sio2Engine

Sio2
Interactive
Open Source http://www.sio2interactive.com/HOME/HOME.html
Oolong

Cannot be
determined
Open Source http://www.oolongengine.com
Raydium 3D Cannot be
determined
Open Source http://raydium.org/data.php
Ston3D for
iPhone

StoneTrip Commercial http://www.stonetrip.com/shiva/publish-3d-game-
on-iphone.html
259
9
th
Unity iPhone

Unity
Technologies
Commercial http://unity3d.com/unity/features/iphone-publishing
Bork3D

Bork3D
Games
Commercial http://bork3d.com
Torque 3D for
the iPhone

GarageGames Commercial http://www.garagegames.com/products/tge/iphone
3.1 The Unity game engine

Having tested a few of the available game engine for the iPhone/iPod Touch, the authors selected the Unity iPhone
Advanced game engine because of its features, ease of use, faster rate of development and deployment of
applications, and its file format support. The Unity game engine is developed by Unity Technologies. Initially, the
Unity game engine was only available on the Mac OS X platform for games development with options to deploy the
games online, on a Windows PC, or on a Mac OS platform. However, recently Unity Technologies releases Unity
2.5 that allows for development on both Max OS X and Windows platforms. The Unity game engine supports the
development and deployment of applications on the iPhone/iPod Touch through the Unity iPhone Publishing (either
Basic or Advanced) software.
The Unity and Unity iPhone Advanced development environment is mostly GUI driven. Like most game engines,
pre-defined modules already exist in the Unity engines. Modules can be dragged and dropped into a project so that
they can be used in a scene. At the very basic level, the modules can be used are they are. However, extended
functionalities can be easily added to any of the existing modules through the use of Java scripts or C# programming
language (see figure 1). These modules can also be imported and reused in other future projects. Figure xx shows an
actual module used in this paper that uses C# programming language. This module sets the screen of the
iPhone/iPod Touch to landscape display mode when virtual environment is displayed.

FIG 1: The SetScreen module uses the C# language

4. DEVELOPMENT PROCESS OF A VIRTUAL ENVIRONMENT APPLICATION ON
THE APPLE IPHONE/ITOUCH

Most modern 3D games are played in virtual environments (VEs). The main components of the VE are terrain,
skybox, lights, 3D assets such as buildings, avatars (to represent player and non-player characters), textures such as
the ones applied to the 3D model, terrain, avatars, skybox etc., audio such as sound effects, environmental sound,
background music etc., game mechanics and physics. The VE that is created on the iPod Touch consists of a model
of a Country House placed on a flat terrain. The game mechanics and physics components of the VE were excluded.
Figure 2 shows an overview of the development workflow. Each development stage is described in the following
sub-sections.
260
9
th

FIG 2: An overview of the development workflow

4.1 Development prerequisites

Development of an iPhone/iPod Touch application (or apps) requires the followings:
1. An Intel-based Mac computer.
2. Mac OS X Leopard operating system software installed.
3. An iPhone or iPod Touch hardware.
4. The iPhone SDK available for free from Apples website.
5. The Xcode software development tool. Xcode is a suite of tools for developing software on the Mac OS X.
Xcode can be downloaded free of charge from Apples website.
6. Join the iPhone Developer Program, and be a licensed Apple developer. A standard program will cost
$99/year. However, by joining the iPhone Developer University Program, Apple will grant free developers
license to any academic institution that will teach iPhone application development as or part of a course.
(Note: Apple recently updated the iPhone OS to version 3.0 in June 2009. Following the new OS release, the iPhone
SDK was also updated to version 3.0. Prior to using the new iPhone SDK, the Mac OS X operating system must be
upgraded to version 10.5.7, the iTunes software to version 8.2, and Xcode to version 3.1.3.)
Once the authors became Apple certified developers, access to the iPhone Developer Program portal website was
granted. Using this portal, a Provisioning Profile and an iPhone Development Certificate were created and installed
onto the iPod Touch, to allow for deployment of new locally developed applications on to it. Since the authors are
using the Unity iPhone Advanced game engine, the Unity Remote application was installed first on the iPod Touch.
The Unity Remote application provides a live preview on the iPod Touch screen by directly linking the Unity Editor
environment on the Mac computer to it. This is a useful feature as it allows for quick testing and debugging of the
application under development.
One of the development intentions is to use any 3D CAD model developed elsewhere and did not originate from the
authors. The reasoning behind this is to test out the level of compatibility of the Unity game engine with respect to
3D CAD model. In this paper, a commercial 3D CAD model of a Country House acquired from BlueBrain3D
(2009) was used as part of the VE.
4.2 Terrain

The author separated the development of the 3D CAD model since it was acquired from BlueBrain3D (2009) . The
development of the VE for the iPhone/iPod Touch began with the creation of the terrain. Unlike the Unity game
engine, the Unity iPhone game engine does not have a dedicated terrain engine; hence terrain can only be created
either using geometries created in another 3D modeling software such as 3DS Max or Google SketchUp, and by
DevelopmentofVE
261
9
th
using a plane geometry (to represent a flat terrain) that is available in the Unity iPhone game engine. In this paper,
since the VE depicts a Country House in a suburban area, the use of a flat terrain was sufficient.
4.3 Skybox

Skies were applied using a skybox method. A skybox is a six-sided cube consists of six-seamless images that
enclose a virtual environment. Each side of the cube represents North, East, West, South, Top and Down
(Shiratuddin et al, 2008, Kanalakis, 2008, Busby et al, 2004). Figure 3 shows the images used for the skybox. A
skybox method is usually employed in computer and video games. This method is very efficient since it will make
the virtual environment looks larger. Skybox can contain skies, mountains, distant objects such as buildings, and any
other objects that are unreachable by the user. A skybox can be created using software like Terragen. Another
method of creating skies in a virtual environment is to use a Skydome, which is a similar concept to Skybox, but
uses a sphere or hemisphere instead. The Unity iPhone Advanced engine supports the rendering of Skyboxes
through the use of the built-in RenderFX/Skybox shader.
4.4 Lights and shadows

Lights and shadows play an important role in a virtual environment. Without proper lighting and shadows, a virtual
environment may look plain and dull. Since the Country House model used in this VE application did not have any
interiors, only exterior sunlight was used to generate shadows. A Directional Light was used to represent sunlight.
A directional light source illuminates a virtual environment using a constant illumination from a single direction.
Since a directional light source it is usually dynamic in nature, and it uses a real-time stencil shadow casting method
(Shiratuddin et al, 2008). However, in the Unity iPhone Advanced engine, shadows created using directional light
sources do not cast real-time shadows; the shadows were baked onto the textures where lights were directed to,
using shadow mapping techniques. This gives the illusion of shadow casting. The shadow mapping technique
increases frame-rate-per-second (fps) in a VE and prevents the engine from rendering shadows in real-time. This
method may not be suitable in virtual environments that require real-time shadows generation e.g. in studies
involving determination of shadow intensities and shadow projections in real-time.
With the terrain, skybox and sunlight in place, the VE is now partially assembled in the Unity Editor. Since there is
no interaction technique implemented yet, the VE is static.

FIG 3: The skybox images used in the VE
262
9
th
4.5 Interaction techniques

Interaction technique is one of the crucial elements in a VE. Some VE applications failed due to improper design
and implementation of good interaction techniques. Poorly designed interaction techniques will make it difficult for
users to interact with the VE. The iPhone/iPod Touch has unique interaction techniques. The two main interaction
techniques are multi-touch and tilt-control movements of the iPhone/iPod Touch itself. For example, in the Google
Earth for the iPhone application, by simply tilting the iPhone/iPod Touch up and down, left and right, a user can
navigate through the world map. In this paper, the authors implemented both types of interaction techniques for the
VE to see which of the two is most suitable for navigation purposes.
Revelle and Reardon (2009) reached several important conclusions when designing learning applications for
children in their research. Many of these principles deal with the unique interaction techniques of the iPhone and the
problems some of the children had with the interface. Simplifying the game structure was one way to help users
from being confused. With reduced screen size and new interaction techniques it is best to keep the game play
simple to avoid unnecessary confusion. Using large, visually distinct hotspots is another of their recommendations.
The screen needs to be as simple as possible with large buttons so that they can be easily selected. Too many buttons
or buttons that are too small quickly become difficult to select without error. Tilt functionality can be tricky for
inexperienced users so starting with a low sensitivity is recommended. Users can accidently activate undesired
commands if tilt functionality is too sensitive. A setting to adjust sensitivity for advanced users might be useful. The
use of audio cues or prompts to describe what actions the user should take can also be helpful. Additional simplified
instructions can be applied after 7 to 10 seconds of being idle if the user is still uncertain of what to do. While these
are not all of the principles described by Revelle and Reardon (2009), they are some of the most pertinent to
navigating a 3D world. These should be considered in the early design phases to avoid problems later.
When first developing the VE, the authors implemented multi-touch interaction techniques (see figure 4). Using this
technique, a user simply uses his/her thumbs to navigate in the VE. 2D navigation compass-like images were
overlaid on the iPod Touch screen and used as the areas where a user can touch to initiate navigation in the VE. The
right compass image once touched, enables a user to look around, and the left compass image allowed forward,
backward, left and right movements in the VE. By touching both left and right navigation compass, a user can
comfortably navigate in the VE displayed on the iPod Touchs screen.

FIG 4: The author using the multi-touch interaction technique to navigate through the VE on the iPod Touch
Next, the authors implemented tilt-control movements. Using this method of interaction technique, a user simply
tilts the iPod Touch in the direction of navigation to move forward and backward, left and right. This technique
however, allows for only navigation in the VE. Looking up and down, left and right could not be combined with this
method of interaction, and this posed a navigation problem on the VE. The authors also discovered that if a user
tilted the iPod Touch too much, large tilting angle made it hard for the user to see the screen. Due to this reason, the
authors concluded that the multi-touch interaction technique is more suitable for navigation in a VE on the
iPhone/iPod Touch. Figure 5 shows a snapshot of Unity Editor loading and at the same time running the VE on the
iPhone simulator application.

263
9
th

FIG 5: The Unity development environment and the iPhone Simulator
5. BENEFITS, CHALLENGES AND CONCLUSION

Some of the benefits of utilizing a 3D game engine, specifically the Unity iPhone Advanced game engine are two-
fold. Applications and games can be developed fairly rapidly since the Unity game engine comes with basic
modules for real-time 3D applications and games development. These modules are highly customizable through the
use of Java scripts or C#. The Unity game engine also has support for major industry standard file formats ranging
from 2D images to 3D CAD models. This feature alone is an immense advantage since developers no longer have to
be concerned with proprietary non-standard file formats.
The benefits of utilizing the iPhone/iPod Touch for 3D architectural and construction visualization are also clear; the
mobility, the sufficient computing power to run 3D real-time applications, the ubiquitous connectivity to wireless
networks, and the unique interaction techniques pose a new but rewarding challenge in creating a visualization
application. For example with the increased mobility brought on by a handheld device, a client can be briefed about
his building over a lunch meeting. Instead of using a laptop, the client can download, visualized, navigate, and
walkthrough the 3D model of the building. This type of application can also be used as a review tool on the job site.
It could also be used in a small meeting to show the end results of proposed project where a projector or computer
may be unavailable or overly cumbersome.
During development, the authors encountered some challenges primarily due to the hardware and software
limitations of the iPhone/iPod Touch. Screen size and 3D scene complexity are perhaps two of the largest
limitations. The small screen provides excellent picture quality but if the display is cluttered with numerous GUI
elements, it can quickly become difficult to click on or begin taking up screen space. The authors have not
thoroughly tested what is the 3D scene size limit the iPhone/iPod Touch can display in real-time. However, as with
other combination of game engine and hardware specifications, this is an expected limitation.
In conclusion, the authors have succeeded in developing a clean and efficient VE that is easy to navigate and walk
through on the iPhone/iPod Touch. The VE application is functional considering the iPhone/iPod Touch size and
Scene assembler window
iPhone Simulator Real-time virtual environment simulator
264
9
th
processing power. The authors future investigation is to test the exact limitations of the Unity game engine and the
iPhone/iPod Touch hardware in detail by comparing frames per second with model complexity and different lighting
conditions as well as various navigation techniques unique to the iPhone/iPod Touch. The authors hope to get
further opinions and comments from architects, design reviewers, project developers, and educators etc., who may
find this type of tool useful in a real world scenario. In short, development for the iPod Touch/iPhone is an ongoing
process that the authors hope will fully utilize the new interaction techniques and 3D processing power of the
devices which make them so unique for development.
6. REFERENCES

Apple (2009a), iPhone Technical Specifications. Website: http://www.apple.com/iphone/specs.html
Apple (2009b), iPod Touch Technical Specifications. Website: http://www.apple.com/ipodtouch/specs.html
Apple Developer Connection (2009), iPhone Developer Program.
Website: http://developer.apple.com/iphone/program/
Block, Ryan (2008), iPhone 3G review. Website: http://www.engadget.com/2008/07/11/iphone-3g-review/
BlueBrain3D (2009), Bluebrain 3D Model Library. Website: http://www.bluebrain3d.com/3d-models/
Busby, J, Parrish, Z. and VanEenwyk, J. (2004). Mastering Unreal Technology: The Art of Level Design
Sams, ISBN-10: 0672326922.
Kanalakis, J. (2008), The Complete Guide To Torque X. AK Peters, Ltd: Wellesley, MA
Moore, J., Oussena, S. and Zhang, P. (2009), A portable document search engine to support off-line mobile learning,
IADIS International Conference Mobile Learning 2009, 26th February 2009, Barcelona, Spain.
Revelle, G. and Reardon, E. (2009), Designing and testing mobile interfaces for children, Proceedings of ACM IDC
2009: The 8
th
International Conference on Interaction Design and Children, pp. 329-332, Como Italy, June
3-5, 2009.
Snell, Jason (2008), That iPod touch runs at 533 MHz.
Website: http://www.macworld.com/article/137139/2008/11/ipod_touch_speed.html
Shiratuddin, M.F, Kitchens, K. and Fletcher, D. (2008). Virtual Architecture: Modeling and Creation of Real-Time
3D Interactive Worlds, Lulu.com, ISBN-10: 1435756428.
265
9
th

3D VISIBILITY ANALYSIS IN VIRTUAL WORLDS:
THE CASE OF SUPERVISOR
Arthur van Bilsen, Ph.D.,
Policy, Organization, Law & Gaming Group
Tel. +31 (0)15 27 87239
a.vanbilsen@tudelft.nl
Ronald Poelman, M.Sc.,
Tel. + 31 (0)15 27 88542
r.poelman@tudelft.nl
http://www.tudelft.nl/rpoelman
ABSTRACT: Technology to create virtual representations is getting more advanced every day. The resolution of
areal datasets is increasing and the reconstruction of those sets is more robust and delivers better models. A lot of
those datasets are freely distributed online such as Google Earth. There are cars equipped with stereo
photogrammetric equipment and laser scanners to acquire even more detailed models of reality. But what can we do
with all this information? The datasets can provide us with more than just virtual worlds, object information and
geometric information. In this paper we shift our perspective from the physical objects to the space in between.
From this space a vast amount of visibility data can be obtained, which only recently has become possible in 3D.
Just as one can scan a real physical environment one can also scan a virtual environment. In doing so, we are
analyzing environments by generating 3D isovists in fully 3D models to provide information other than the basic
object and geometric aspects. This information may unlock many possible uses, such as to indicate human activity
and cognitive load in public spaces, to provide security surveillance information and to quantify the diversity of the
living environment. The research suggests ways to use isovists and shows an application thereof in a training
simulation game about safety.
KEYWORDS: Virtual, scanning, visibility, analysis, 3D-isovist.
1. INTRODUCTION
1.1 Virtual environments
There is an increase in use of 3D virtual environments in the architecture, engineering and construction. Many man
hours are spend on the modelling processes for the creation of these environments. Although 3D virtual
environments are enhancing understanding of the design process, they can be used for other types of understanding
as well. These past years, ample attention was directed at graphics. Especially realistic on-line rendering benefitted
greatly from this trend. Most noticeable are the developments in the entertainment gaming industry, where vast
amounts of new titles appear with appealing visuals. The industry is currently transferring their visual effects from
off-line to on-line rendering (Akenine-Mller and Haines, 2002), which means photorealism is brought within reach
of large online environments in real-time.
When looking at recent entertainment game titles, it becomes apparent that the complexity embedded in the virtual
environments is also increasing. To satisfy the gamers with ever more immersive games, entertainment games
display complex weather systems, vegetation systems and artificially intelligent wild-life. And also fire systems are
being developed to mimic actual spread of wildfire. These are all systems based on known simulation models that
were subsequently stripped, to be able to run in real time. On the visual side, realistic lighting and shadow options
are also represented based on high dynamic range lighting, a technique which aims to increase visual realism. A
further constraint that is being relieved, is the number of polygons and textures that can be used in a virtual game
267
9
th

environment. Together, these developments support simulation of complex systems and their visualisation in greater
detail than before.
Not just visualisation is benefitting from technological developments, the process of capturing reality is also making
leaps. Reality gets captured and measured more frequently and denser than ever before. The acquisition of areal
datasets, which span most cultivated areas on Earth, doubled in resolution the past year. And because of better areal
datasets the reconstruction quality is also improving. To name a few, individual lucarnes, windows and chimneys
can already be reconstructed from areal laser altimetry and stereo photogrammetry. Parallel to areal-based data
acquisition there is an increase in terrestrial acquisition; laser scanning is getting more regularly used and stereo
photogrammetry benefits from both computer speed and better sensors. Both laser scanning and stereo
photogrammetry systems can be mounted on vehicles to facilitate quick acquisition.
Although it still takes a lot of time to create realistic 3D environments, the process is also becoming more automated
due to clever algorithms. Nevertheless, the computer is still not capable of creating the structured detailed
environments that feed directly into advanced visualisation engines, without some manual human intervention. For
realistic visualisation of environments all kinds of additional information is stored in addition to the basic geometric
information. A contemporary entertainment game title has at least 5 layers of texture information attached to its
environments and characters. There is a diffuse texture for colour, a light map for the storage of lighting
information, displacement and normal maps for additional detail and a specular map for light fall-off. It is these
maps and the developments in GPU programming that have made 3D visibility analysis possible. The maps are for
example used to store isovist fields, which are shown in the figures of section 3.
1.2 Visibility analysis
Ever since the dawn of sighted life on earth, seeing and being unseen have been key survival skills. Both the hunter
and the prey need to take visibility into account as do two potential mating partners. The hawk benefits from an
overview, the lion sneaks up as invisible as possible on prey. Other animals, such as those that live in the ground or
in trees, construct their hidden habitats based on visibility considerations (Hall, 1966).
Visibility analysis can be seen as a shift in focus from the physical objects to the space that separates them. Where
the focus on objects brings efficient ways of storing geometrical and material properties, this shift has other benefits.
Space is where people reside, move about and employ activities. Virtual space is where the players, avatars, agents
and opponents are, move about and demonstrate their behaviour. While objects may contain information, such as
geometrical information, the shift opens up richer ways to investigate the relation between the observer and its
spatial environment. This relation is rooted in the survival value, the environment represents. While humans attempt
to explore and understand their environment (Kaplan and Kaplan, 1989), visual information is their richest source of
information, e.g. in terms of bandwidth, and it is not unlikely that sight has therefore become the dominant sense
(for those that have it). Apart from a wealth of colour information, sight also provides proximity information, both
of which have obvious survival and reproduction advantages. However, light as a carrier of proximity information,
is not the only solution stumbled on by nature. Bats use sound waves in a process called echo-location to generate a
kind of depth-array of their surroundings, just like sighted animals (Gibson, 1987). This depth array is not only
useful for animals, but also for humans.
The process of perception is more than just looking. It also involves walking around objects and touching them.
Hence a more complete representation of the 3D environment is formed, than one would expect from a simple
directed camera. Apart from peripheral vision, the latter is another argument for the use of the full 360 degrees view,
instead of just a narrow forward view. Even though one may be looking forward at one particular moment, in the
process of moving and looking around a more complete representation of the environment is usually present in the
brain. This representation returns later in section 2.5 as the concept of isovist, where it aids in formalizing visibility
analysis. An isovist is a depth array, or simply what you can see from a single vantage point.
Visibility analysis of mans habitat: architectural and urban environments, was formalized first by Benedikt (1979).
He introduced visual characteristics, called isovist measures, such as visible area, perimeter, variation and skewness.
Note that visibility works in two directions. If you have more of your surroundings in view, there are potentially
more places from which you can be seen by others. Privacy is an example where this is relevant. It is achieved at
observation points with a good view of the surroundings and low angular exposure. In terms of isovist measures this
corresponds to high area and skewness (ibid.). Earlier Newman (1973) had already found that it is more likely for
268
9
th

crimes to be committed in spaces with poor visibility from the surroundings, as thieves will generally try to remain
hidden from sight. From these early observations one finds that visibility analysis can help the design or redesign of
buildings and public spaces.
The mathematical framework used in this paper, builds on Benedikts framework. In line with Batty (2001, p.124),
our main step is not to try and summarize space as do Peponis, Wineman, Bafna, Rashid and Kim (1998a; 1998b),
but to work with the full visual field at each vantage point. The mathematical foundation for visibility analyses was
laid by Benedikt (1979). It was extended by Van Bilsen (2008) and coined Isovist-based Visibility Analysis (IBVA).
However, the actual calculation of visibility measures in practical applications remained a hard problem. Early
applications were for the majority conducted on a 2D map and this is still the dominant use today. In the
development of three-dimensional measures, first steps have been taken: spatial openness by Fisher-Gewirtzmann
and Wagner (2003), sky opening by Teller (2003), real world measurements of the latter by Sarradin (2004) and
more recently various measures in digital elevation models (DEMs) by Morello and Ratti (2008). However none of
these can be considered a general three-dimensional approach. For example, DEMs cannot adequately represent
cities when it comes to observer experience of inside spaces (roofed) and outside spaces (arches, bridges, etc.).
Recently, due to the rapid increase in calculation and memory capacity of computer hardware, full three-dimensional
analyses have become feasible (Van Bilsen, 2008, 2009). We argue that some of the first results can support the
construction of virtual game worlds in new ways and an example from a serious game environment is elaborated on
in this paper.
Virtual environments have the potential to fully immerse players, not in the least because of high resolution images,
and realistic dynamic lighting, fog, blur and texture effects. The techniques behind these effects can be automated,
and are being used by the authors to efficiently analyse virtual environments on visibility characteristics of which an
example is given in this paper. We will describe some methods and principles and their relation to visibility analysis.
2. METHODS AND PRINCIPLES
Visibility analysis concepts overlap a great deal with ideas and concepts from related disciplines and vice versa. For
clarity, in this section we position visibility analysis in relation to laser scanning, spherical harmonics, ambient
occlusion and visibility graph analysis (VGA). We will see that they have many common basic notions. This overlap
is an opportunity for synergy among until now mostly separated disciplines. The similarities are remarkable, as for
example the main difference between laser scanning and isovist generation is that one is performed in the real world
while the other takes place in a virtual world (possibly modelled after the real world). Due to the restrictions of this
paper we are not able to provide an exhaustive overview and have selected some with high synergy.
2.1 Laser scanning
A laser scanner is a device that analyzes real-world environments and objects. It collects data on the geometric
shape, but allows acquisition of other data, such as reflectance properties and normal direction. There are different
laser scanning methods for 3D data acquisition. Most common methods are based on triangulation, time-of-flight
and phase-shift. A frustum is captured or even a full sphere with a grid of laser measurements. As laser scanning
measurements use light, it is not possible to get information behind the first obstruction a ray of light encounters.
Hence it is a line of sight measurement technique. Consequently it generally requires multiple scans to capture a
complete object or environment.
The capturing process is very much like the creation of isovists although the main difference is evident; the
capturing of reality versus virtual reality. Where laser scans have to deal with atmospheric conditions and limited
range, virtual scans have fewer constraints. Although lasers have existed since the 1960s, the use for mid-range laser
scanning is relatively new (Frhlich and Mettenleiter, 2004). To highlight the connection with isovists a type of
scanner that most resembles an isovist will be highlighted.
2.1.1 Phase-shift spherical scanners
There is an increasing attention for phase-shift scanners because of high speed and dense scanning capabilities. They
can measure more than half a million points per second and provide spherical datasets. The laser rotates vertically
with thousands of revolutions per second creating a vertical slice of the scenery in each rotation. Synchronously, it
rotates horizontally to change the angle of each slice. The resulting dataset is thus actually a set of vertical slices
269
9
th

with the origin being the scanner. These directions also correspond to the isovists spherical coordinate directions
and .
Although scanning seems an ideal way to capture the environment, there are some disadvantages which are
(partially) shared with isovists. A full dome will approximately take 2 minutes, within that time the condition of the
environment might have changed such as lighting conditions or moved objects. In general, virtual environments do
not suffer from this limitation as their time can in principle be stopped. Obvious exceptions are real-time online
environments. The inclination angle of the laser will affect the accuracy; scanners have a footprint which is based on
the diameter of the laser beam. Measurements that are more parallel to the laser will have the shape of an ellipse the
centre of which is used to get depth information. This naturally affects accuracy: surfaces that are at a 90 degree
angle with the laser are not affected by the inclination angle. Furthermore, some colours in the light spectrum will be
absorbed or affect the laser negatively, such as dark colours and some red tones. Finally, the range of this type of
scanner is quite limited, in general 50-100 meters, due to its set phase-shift principle and eye safety regulations.
Isovists can be generated in virtual worlds over much greater distances, limited by floating point constraints.
2.2 Spherical harmonics
Spherical harmonics are used to create realistic real-time lighting in complex 3D scenes. Essentially, the scenes
lighting information is stored in textures of spheres which can be updated when lighting conditions change.
Sampling the scene this way is similar to the use of isovists, although the information they store is quite different.
Formally, spherical harmonics are the angular portion of a set of solutions to Laplace's equation, represented in a
system of spherical coordinates. Laplace's spherical harmonics are a specific set of spherical harmonics which
form an orthogonal system.
In 3D computer graphics, spherical harmonics are used for indirect lighting such as global illumination, ambient
occlusion and in pre-computed radiance transfer (Green, 2003). Although large amounts of textured polygons can be
rendered within a second using state-of-the-art graphics cards, rendering a realistic complex scene at real-time speed
remains difficult. With spherical harmonics the rendering primitives are no longer geometrical entities, but sampled
images wrapped around spheres. The scene is first sampled from different viewpoints and under different
illuminations. Spherical harmonics work at best, and are especially efficient, when illuminated with directional light
sources. Whenever the user changes the illumination setting, only a certain number of views have to be
reconstructed. Information is stored, such as light direction and light intensity. Although it is a relatively new
method to light real-time scenes, it allows image-based objects to be displayed under varying illumination.
2.3 Ambient occlusion
Ambient occlusion is a shading method which adds realism by taking into account the attenuation of light due to
occlusion. Ambient occlusion is an approximation process that attempts to simulate the way light radiates in real
life. (Langer and Blthoff, 2000) It is a global method because the illumination at each point is a function of other
geometry in the scene. The effect of ambient occlusion is similar to an objects appearance on an overcast day which
is generally used as a composition layer on 3D scenery. The equivalent of ambient occlusion in the IBVA
framework is one minus the sky factor (see FIG. 4).
Because ambient occlusion is using only the geometrical shape of the environments to create its maps it is therefore
related to the creation of isovists. Although an isovist calculates a lot more parameters and has full flexibility in the
appearance of its parameters there is certainly overlap in the mathematical approach. Ambient occlusion is mostly
calculated by casting rays in every direction from the surface. Rays which reach the background or sky increase
the brightness of the surface but a ray that hits any other object does not contribute to illumination. The result of this
type of calculation is that points surrounded by a large amount of geometry are rendered dark and points with little
surrounding geometry are light. The maps that are calculated by ambient occlusion are generally stored in the UV
space of the textures. It can be visualized as a displacement map or colour texture map.
2.4 Visibility graph analysis
In the early 80s, an approach slightly different from Benedikts approach emerged. Hillier coined the representation
of urban space as a matrix of the longest and fewest lines, the axial map', based on visibility graph analysis
(VGA). He also introduced the use of the various versions of the topological (i.e. non-metric) measure of patterns
270
9
th

of line connectivity called integration (Hillier et al, 1982; Steadman, 1983; Hillier & Hanson, 1984). Where IBVA
and Benedikt start with continuous space, space is now turned into a visibility graph, which has a discrete nature.
This set of analysis methods is known as Space Syntax and one typically uses a 2D map, with polygonal occluders
as buildings. Space Syntax has generated not only models for predicting urban movement (Peponis et al, 1989) but
has also yielded a practical method for the application of these results and theories to design problems which has a
substantial portfolio of projects (Hillier, 1997).
A significant correlation is found between principal lines of sight and pedestrian movement (Hillier, 1998; Carvalho
and Batty, 2003). These so-called axial lines can be derived algorithmically from certain isovist-fields (ibid.). It was
found (de Arruda Campos, 1997) that the number of axial lines (principal lines-of-sight) intersecting at a square,
correlates with the number of people making informal use of the square. Also a correspondence was noted (Conroy,
2001) between junctions, particularly with long lines of sight and large isovist area, and places where people pause
on journeys. The approaches of VGA work best in urban environments, where there are many occluding buildings
and medium to long streets. However, for more open spaces, the axial lines say less about the behaviour of
pedestrians and the method is no longer applicable in these environments. The virtual environment analysed in this
paper is rather open and illustrates a case in which VGA is not applicable.
2.5 Isovist-based visibility analysis (IBVA)
This method represents a way of conducting visibility analysis based on the concept of an isovist (Benedikt, 1979).
It supports systematical formalisation and application of visibility analysis to real and virtual worlds. As stated in the
introduction of visibility analysis (1.2), the first step is not to try and summarize space as do Peponis, Wineman
Bafna, Rashid and Kim (1998a; 1998b), but to work with the full visual field at each vantage point Batty (2001,
p.124). The mathematical extension towards 3D was described by Van Bilsen (2008). Part of the evolution of
visibility analysis as a science, is a proper error analysis for existing 2D analyses (Van Bilsen and Stolk, 2008),
since in general the quality of scientific results is strongly related (inversely proportional) to the error made.
Although highly focussed on proximity, visibility analysis is broader, as it also includes colour, intensity,
information processing and perception of what is seen (Gibson, 1987; Van Bilsen, 2008). Empirical correlations
between several measures can be found in the work of Stamps (2005) and Stamps and Krishnan (2004). IBVA is
used in the current paper, because a mathematically founded 3D analysis is needed for real and virtual world
applications in general, as well as for the Supervisor training environment example below.

FIG. 1 A 2D isovist (blue area, left) and a 3D isovist (volume within blue-beige sheet, right).
From: Van Bilsen and Stolk (2007).
In simple terms, an isovist is What you can see from a single vantage point (FIG. 1) as mentioned earlier. More
formally, an isovist is a Euclidian, geometrical representation of space (a connected region), as can be seen from one
observation point bounded by occluding surfaces. The region of space an isovist encompasses has a volume, an
outer boundary area and many geometrical characteristics. In a small room, with curtains closed, ones isovist
volume is small, but out in the open field, ones isovist volume is enormous, solely bounded by ground, horizon and
sky. By calculating visual characteristics, based on the distances to the environment, at a large number of vantage
points in space one obtains a scalar field, which is visualized as a heat-map or height-map. The figures in this paper
show these planes, called isovist fields, cutting through 3D objects and space.
271
9
th

3. SUPERVISOR
A large oil company requested a pilot showing the benefits of serious gaming for the training of supervisors. The
resulting training game, Supervisor, can be classified as a serious game. Serious games are simulation games that
use technology from entertainment gaming. For creating Supervisor, the Unreal Engine 3 was used (Epic, 2007). In
the early stages of the design process, already the decision was made to create an immersive 3D serious game with
various scenarios, which need to take place in virtual versions of actually existing drilling sites. Because immersion
was considered an important aim of the game, the virtual environment had to at least be realistic. In the game,
supervisors are being trained more specifically in health safety and environment (HSE) tasks. One of their tasks is to
have a good visual overview of the site. Although this task can be trained by actually walking around the site
physically, isovist analysis of the scene can help spot suitable locations on the site, via the virtual version.

FIG. 2 A view inside Supervisor (left) with references to FIG. 9. The model of the virtual world of Supervisor, which
was used for the visibility analysis, stripped of textures for efficiency (right).
Because supervision and in general HSE are of great importance to the client, this case was selected for this paper.
Our aims with the visibility analysis of Supervisors virtual world are: to assess the degree to which quantitative
visibility data provides relevant information for virtual world design. Since Supervisor was purposefully designed
for learning about safety, specific attention will be given to safety aspects.
3.1 Isovist-based visibility analysis of the virtual world of Supervisor
In applying IBVA to the virtual world of Supervisor we try to assess the relevance of quantitative 3D visibility
analysis for design. The results, however, hint towards further conclusions about the use and possibly the regulations
of an environment with regard to safety. Any experimental result is highly influenced by the variables that one
chooses to measure. We have not applied a selection criterion on the set of measures available. They are measured in
the 3D model and use distance data only. Some may only use the distance data in a horizontal plane at some height
(e.g. 160cm) and are called 2D, keeping in mind the environment is 3D. Other measures use the distances in all
directions, covering the entire sphere around the observer, and end with 3D. The distances are limited to 225m
since this includes the complete environment from every observation point. Here is the list of measures that were
calculated at each observation point (TABLE 1).
TABLE 1 Long list of measures calculated. Selected measures for this paper have an *.
AREA-2D* The isovists area in the horizontal plane
AVGR-2D The average distance to the environment, in the horizontal plane
AVGR-3D* The average distance to the environment
MAXR-2D The maximum distance to the physical environment, in the horizontal plane
MAXR-3D The maximum distance to the physical environment
MINR-2D* The minimal distance to the physical environment, in the horizontal plane
MINR-3D* The minimal distance to the physical environment
PER-2D* The isovists perimeter, in the horizontal plane
SKY-3D* The solid angular fraction of sky visible
STD-3D* The standard deviation of the distance to the environment (= )
272
9
th

In addition, also the horizontal spatial gradient of these measures was calculated. The mathematical definitions of
these measures are taken from Van Bilsen (2008). The measures are calculated on a rectangular spatial grid of about
80,000 observations points slicing through the Supervisor world model of approximately 200m x 100m. The
distance between two neighbouring observation points is therefore approximately 0.5 meter, the length of a short
step of an adult. The grid resides in a horizontal plane at some height. A plane at 160cm was chosen to observe
visibility characteristics. This height holds the middle between average eye-height levels while standing up, kneeled
down and bending over. Another plane was chosen at 20cm for comparison.
The isovist generation and isovist measure calculations were done using software from Aisophyst
, according to the
following specifications. At each observation point 1,566,726 lines of sight (distances to the environment) are
traversed, pointing in all directions. The precision of each distance is floating point: 10
-7
. At increasing value of the
distance, the number of significant decimals will decrease. The minimum distance is 10cm. As the maximum
distance is 225m, the error is certainly never larger than 0.5cm, and will typically be lower. Each line of sight covers
a concomitant finite solid angle (on average 810
-6
sr) of which the relative error is 1%. Therefore all measures
calculated in this paper have a relative error of at most 1%. The data visualisations shown in this paper, and the
spatial gradient calculations were obtained with MATLAB. All isovist measures are calculated in a plane at
160cm unless stated otherwise.
3.1.1 Results
Safety, the supervisors overview and equipment
A large average distance to the environment, AVGR-3D, is a good indicator for the amount of accessible volume
around an observer. Volume may indicate both available distance as well as visible area (orthogonal to lines of
sight) and does not discriminate between the two. Thus it should not be used for detailed results, but may e.g. shed
light on the perception of the amount of surrounding transparent volume. Nevertheless, the places where both high
AREA-2D and AVGR-3D (FIG. 3) are high are good candidates for a supervisor that needs an overview. At drilling
sites, safety equipment such as fire extinguishers should be placed well in sight. And there should always be enough
room for fire trucks and ambulances to enter the site. Below, further analyses will be able to provide more detailed
information. The black line arbitrarily separates areas where the average distance is higher or lower than 100m. It
shows that particularly the larger structures, such as buildings, and not the smaller objects, such as benches,
influence available volumetric space.

FIG. 3 The isovist measure AVGR-3D (left) in a 225m sphere. The black line separates places with values below and
above 100m. The isovist measure AREA-2D (right). The black line separates places with high and low value (low is
dark blue, see legends).
In drilling site terminology, the monster spot is chosen at a safe location on the terrain: it needs to have cover, one
should be able to count each other, and alternative meeting places should be accessible nearby. In FIG. 3 (left) the
monster sport would be chosen outside the black line. In combination with the constraint of being able to stand at
least 2 to 3 meters away from objects on the ground (20cm above ground), FIG. 8 (left) tells us there are three
candidates for the monster spot: at the top left corner, at the top right, and at the lower middle part of the site.
The area in the horizontal plane indicates the extent to which one has possibilities to see ones environment in the
horizontal direction at 160cm above the ground: AREA-2D. Within the black lines one has less visible area than
outside the black line. Subsequently, outside the black line there is more overview. This measure may also indicate
273
9
th

the amount of escape area available, although objects lower or higher remain undetected by this measure and may
act as obstacles. This is analysed below by comparison of the 20cm and 160cm isovist fields of MINR-2D.
Safety through good natural lighting conditions and shelter
It takes a small amount of time for the human eye to adapt to sudden changes in lighting conditions. Although it still
beats photo and video cameras, within seconds the environment may have changed without notice, such as stacked
crates tumbling over (one of the cases designed into Supervisor for players to prevent). The figure highlights places
where the natural lighting conditions change suddenly, by showing the gradient of SKY-3D (FIG. 4, left).

FIG. 4 The isovist measure gradient of SKY-3D (left). High values indicate a change of natural lighting
conditions. The isovist measure SKY-3D (right). The black line encloses places with low lighting (see legend).
Natural light comes from the blue or clouded sky and the sun. The measure SKY-3D (FIG. 4, right) does not
discriminate between these, but merely provides the amount of sky visible to a particular observation point,
measured in steradians. Nevertheless, it still summarizes the potential of natural light arriving at a point. The black
line separates places where more and less than 30 percent of the total sphere around the observer, or in other words
60 percent of the skys halfsphere is visible. Within the black lines, the darker areas have poorer natural lighting
conditions and are therefore candidates for artificial lighting. As the sky is also the source of various hostile weather
conditions such as snow, rain and wind, the measure is also an indicator of the absence of shelter against these
conditions. In this manner, the design of both shelter facilities and an efficient lighting distribution may be aided by
the SKY-3D isovist field. Note that, the opposite of SKY-3D, for example (1SKY-3D) signifies the amount of
physical environment in view (called ambient occlusion in section 2.3). This can support pursuit of optimal camera
placement. Also finding optimal surveillance paths for security guards and law enforcers are among the possibilities.
Safety and information overload
There is a limited amount of visual information that can be processed by the brain in a particular period of time.
Early research suggests the bandwidth is about 5 to 9 bits per second (Miller, 1956). The more complex and
extensive an environment, the longer it takes to comprehend it. As noted earlier, the process of perception is more
than just looking. It also involves walking around objects and touching them. This process may take minutes, hours,
days or weeks, depending on the complexity and size of the environment.

FIG. 5 The isovist measures STD-3D (left) and gradient of STD-3D (right).
274
9
th

FIG. 6 The isovist measure PER-2D (left) and gradient of PER-2D (right).
An isovists perimeter, PER-2D, and the (square root of the) variance, STD-3D, may indicate visual information
loads, as these measures are sensitive to fluctuations in the distance to the environment (recall FIG. 1). The latter,
STD-3D in FIG. 5 (left), shows a smooth field changing rapidly only when approaching large objects such as
buildings. The perimeter (FIG. 6, left), however, provides a more detailed indication of places with high information
load. The analysis shows the wealth of depth fluctuation caused by the drilling platform in the middle of the site,
which is made of fine-grained steel frames and contains many metal tubes and pipes. Potentially, people visiting the
site for the first time, will need time to take in the complexity of the structure (FIG. 2, right), they may stand still
suddenly or have less attention for dangerous situations developing in their proximity. From a learning and training
perspective, information overload is best avoided, and a more gradual introduction of information is usually
preferable. For virtual world designers, these indicators can support finding a suitable information load and hence a
desired learning curve.
Safety through a safe 3D distance and safe paths
The safest paths available to a fly at 160cm are not always safest to a walking human, who has to move the entire
body from head to toe. The toe may encounter obstructions not present at the head level and vice versa. When
choosing as safest path the path that leads a pedestrian farthest from obstacles she could collide with, the path
coincides with a Voronoi diagram. The lines of this diagram occur where the gradient of MINR-2D is zero in at least
one direction. The figures show the differences of the Voronoi diagrams at 20cm and 160cm above the ground. In
the Supervisor world, safest paths at 20cm (for your walking feet) are different from the safest paths at 160cm.

FIG. 7 The isovist measure gradient of MINR-2D (Voronoi) at two heights: 20cm (left) and 160cm (right).
Distracted or running people will have to watch the ground in front of them to prevent collisions with low
obstructions. The same people will also have to watch their heads, as obstructions may e.g. hang from above. In
order to make the differences visible first the minimal distance to the environment is shown at two heights. The
majority of obstructions are low, but also some reside higher up. This is shown more clearly by subtracting the two
fields: S = MINR-2D
160cm
MINR-2D
20cm
shown in FIG. 9 (left). Places where there is more proximity at 160cm
than at 20cm have positive values, of which a subset is painted white. Places where there is more proximity at 20cm
than at 160cm have negative values, of which a subset is painted black. The white areas, excluding FIG. 9s
boundary line, indicate danger of tripping over obstacles. The black areas indicate where one is in danger of banging
ones head, such as the stairs shown in FIG. 2 (C).
275
9
th

FIG. 8 The isovist measure MINR-2D at two heights: 20cm (left) and 160cm (right).

FIG. 9 The isovist measure S (left) with the cables (A), pipes (B) and stairs (C) shown in FIG. 2, and the gradient of
S (right), where S = MINR-2D
160cm
MINR-2D
20cm
.
Three black areas are places where steel cables for stabilizing the drill tower, are pinned in the ground (FIG. 2, A). A
careless pedestrian could easily run into such a cable. The cables are visible in the MINR-2D
160cm
figure above as
paired spots. Nearby white areas indicate the pins used to secure the cables in the ground. The pipe storage in FIG. 2
(B) is identified with a white boundary indicating tripping danger.
The gradient of the subtraction in FIG. 9, (right) shows where the difference between heights occurs most sudden.
The figure highlights dangerous spots, where collisions, tripping and bumping may lead to serious accidents. In
other words, it shows the intensification of obstructions one can collide with. Alternatively, we could have used the
3D version of MINR at 160cm, which also shows the proximity of obstacles below and above 160cm. From this
analysis the sites layout is most easily identified.

FIG. 10 The isovist measure MINR-3D. The black line marks intersection with a 1 meter proximity sphere.
276
9
th

The value of such analyses is found in the support of not only redesign of the physical environment but also for
procedures to structure safe use. Although the conclusions are still preliminary, the analyses bring about the
potentials and weaknesses of a virtual world, whether it is modelled after a real world or not. Possible design flaws
and suggestions for improvement can be calculated and evaluated before a design is realised, shortening the
realisation phase and preventing costs of having to correct design flaws afterwards.
4. CONCLUSIONS
The aim of the visibility analysis of (a model of) the virtual world of Supervisor was to assess the degree to which
quantitative visibility data provides relevant information for virtual world design. The set of calculated isovist
measures, albeit a small selection of all thinkable measures, were connected to safety issues. They were shown to be
relevant to real world designers as well as to virtual world designers.
From the isovist fields, determining the best places for a supervisor, i.e. with optimal overview, was possible by
combining multiple fields: high AREA-2D and high AVGR-3D. In addition, three suitable locations for the so-
called monster spot were identified, based on AVGR-3D and MINR-2D
20cm
. Another practical application was
suggested in the placement of safety equipment in spots that need to be well visible. Areas were identified where
lighting and shelter (for weather) conditions are poor or change rapidly by calculating isovist measure SKY-3D and
its gradient. The design of an artificial lighting system could be optimized based on this data.
Identifying the spots where information overload may occur, was done based on PER-2D and STD-3D, although it
must be kept in mind that these do not calculate information but indicate fluctuations in the depth field, which is a
good indicator for information. The analysis shows that the complex metal frames, pipes and tubes of the build
structure in the middle of the site, causes a potential increase in information load. Safest paths should be based on
proximity measurements (e.g. Voronoi) at heights below a pedestrians centre of gravity to prevent tripping or
collision. By subtracting isovist fields from two different heights (20cm and 160cm), dangerous places were
automatically highlighted white (for tripping) and black (for bumping ones head). Alternatively, the 3D isovist
measure MINR-3D yields a similar result directly, although the direction (e.g. obstacle is low or high) is lost,
because this was not stored.
Our future research may develop in several directions. Isovist measures can help explain the behaviour of humans,
by quantifying and correlating the characteristics of the space the live in. As a consequence IBVA also supports
behaviour modelling of artificial actors in virtual environments, such as agents, opponents, and robots. Building
robots can be trained and tested for automated construction behaviour. Artificially intelligent behaviour models can
be supported by much more detailed visibility data than before. In any case, the visibility analysis of spatial
environment yields a vast amount of data, which may contain support or falsification for existing and future
hypotheses. Which hypotheses can be tested remains an open question. Nevertheless, visibility is a general notion
and it can work as a connecting concept across soft and hard disciplines.
5. REFERENCES
Akenine-Mller, T. and E. Haines (2002). Real-Time Rendering. Second edition. A. K. Peters Ltd.
Batti, M. (2001). Exploring isovist fields: space and shape in architectural and urban morphology. Environment and
Planning B, Vol. 28, 123150.
Benedikt, M. L. (1979). To take hold of space: Isovists and isovist fields. Environment and Planning B, Vol. 6, 47
65.
Carvalho, R. and M. Batty (2003). A rigorous definition of axial lines ridges on isovist fields. Casa Working Paper
Series, No. 69.
Conroy, R. (2001). Spatial Navigation in Immersive Virtual Environments. PhD Thesis, Bartlett Faculty of the Built
Environment. London, University College London.
de Arruda Campos, M. B. (1997). Strategic Space: patterns of use in public squares of the city of London. First
International Symposium on Space Syntax, London.
Epic Games (2007). Unreal Engine 3. www.epicgames.com.
277
9
th

Fisher-Gewirtzmann, D. and I.A. Wagner (2003). Spatial openness as a practical metric for evaluating built-up
environments. Environment and Planning B, Vol. 30.
Frhlich, C. and M. Mettenleiter (2004). Terrestrial laser scanning new perspectives in 3D surveying.
International archives of photogrammetry, remote sensing and spatial information sciences, 8/W2.
Gibson, J. J. (1987). The Ecological Approach to Visual Perception. Lawrence Erlbaum Assoc.
Green, R. (2003). Spherical Harmonic Lighting: The Gritty Details. GDC 2003, San Francisco.
Hall, E. T. (1966). The Hidden Dimension. New York, Doubleday.
Hillier B, (1997). The Hidden Geometry of Deformed Grids: or, why space syntax works when it looks as though it
shouldnt. Space Syntax First International Symposium, London.
Hillier, B. (1998). The Common Language of Space. www.spacesyntax.org.
Kaplan, R. and S. Kaplan (1989). The experience of nature: A Psychological Perspective. Cambridge, Cambridge
University Press.
Langer, M. and H. Blthoff (2000). Depth discrimination from shading under diffuse lighting. Perception, Vol. 29,
649660.
Miller, G. A. (1956). The Magical Number Seven, Plus or Minus Two: Some limits on Our Capacity for Processing
Information. Psychological Rev., Vol. 63, 8197.
Morello, E. and C. Ratti (2009). A digital image of the city: 3D isovists in Lynch's urban analysis. Environment and
Planning B. Advance online publication, doi:10.1068/b34144t.
Peponis J, Wineman J, Bafna S, Rashid M, and S.H. Kim (1998a). On the generation of linear representations of
spatial configuration. Environment and Planning B, Vol. 25, 559576.
Peponis J, Wineman J, Bafna S, Rashid M, and S.H. Kim (1998b). Describing plan configuration according to the
covisibility of surfaces. Environment and Planning B, Vol. 25, 693708.
Sarradin, F. (2004). Analyse morphologique des espaces ouverts urbains le long de parcours. cole Doctorale,
Mcanique, Thermique en Gnie Civil. Nantes, Universit de Nantes. Doctorat: 224.
Stamps, A. E. I. (2005). Visual permeability, locomotive permeability, and enclosure. Environment and Behavior,
Vol. 37, No. 5, 587-619.
Stamps, A.E.I. and V.V. Krishnan (2004). Perceived enclosure of space, angle above observer, and distance to
boundary. Perceptual and Motor Skills, Vol. 99, 11871192.
Teller, J. (2003). A spherical metric for the field-orientated analysis of complex urban open spaces. Environment
and Planning B, Vol. 30, 339356.
Van Bilsen, A. (2008). Mathematical Explorations in Urban and Regional Design. PhD Thesis, Delft University of
Technology, Netherlands. Available online: search for authors name Arthur van Bilsen in Google.
Van Bilsen, A. and E. H. Stolk (2007). The Potential of Isovist-Based Visibility Analysis. The Architectural
Annual 20052006 (Bekkering H. C., Klatte J. and D. Hauptmann, editors), Rotterdam, 010 Publishers.
Van Bilsen, A and E.H. Stolk (2008). Solving Error Problems in Visibility Analysis for Urban Environments by
Shifting From a Discrete to a Continuous Approach. IEEE Proceedings of ICCSA 2008, June 30July 3,
Perugia, Italy.
Van Bilsen, A. (2009). How can serious games benefit from 3D visibility analysis? Proceedings of ISAGA 2009,
June 29July 3, Singapore.
278
9
th

EVALUATION OF INVISIBLE HEIGHT FOR LANDSCAPE
PRESERVATION USING AUGMENTED REALITY
Nobuyoshi Yabuki, Ph.D., Professor,
Division of Sustainable Energy and Environmental Engineering, Osaka University;
yabuki@see.eng.osaka-u.ac.jp and http://www.y-f-lab.jp
Kyoko Miyashita, Graduate Student,
miyashita@it.see.eng.osaka-u.ac.jp
Tomohiro Fukuda, Dr.Eng., Associate Professor,
fukuda@see.eng.osaka-u.ac.jp
ABSTRACT: Preserving good landscapes such as historical buildings, temples, shrines, churches, bridges, and
natural sceneries is important. Recently, the number of cases of destroying landscape from view point fields due to
construction of high rise buildings is increasing. To regulate structures such as high rise buildings and high voltage
transmission towers, the government or public agencies have established or are going to constitute height
regulations for building and structures surrounding the landscape target. In order to check whether some portions
of high structures are visible or not behind the target objects from multiple view point fields, it is necessary to make
a 3D model representing geography, existing structures and natural objects using 3D CAD or Virtual Reality (VR)
software. However, it usually takes much time and cost to make such a 3D model. Thus, in this research, we propose
a new method using Augmented Reality (AR). In this method, a number of 3D Computer Graphics (CG) rectangular
objects with a scale are located on the grid of 3D geographical model. And then, the CG models are displayed in an
overlapping manner with the actual landscape from multiple view point fields using the AR technology. The viewing
user measures the maximum invisible height for each rectangular object at a grid point. Using the measured data,
the government or public agencies can establish appropriate height regulation for all surrounding areas of the
target objects. To verify the proposed method, we developed a system deploying AR Toolkit and applied it to the
Convention Center of Osaka University, deemed as a scenic building. We checked the performance of the system
and evaluated the error of the obtained data. In conclusion, the proposed method was evaluated feasible and
effective.
KEYWORDS: Augmented reality, landscape, preservation of landscape, building height regulation
1. INTRODUCTION
Preserving good landscapes such as historical buildings, temples, shrines, churches, bridges, and natural sceneries is
important. Recently, a number of cases showing the destruction of good landscapes from view point fields due to
construction of high rise buildings have been reported. Figure 1 (a) shows a picture of Phoenix Hall of Byodoin, Uji,
Kyoto, which was constructed in 1053 and is a world heritage. Behind the Hall, a tall condominium building can be
seen and it apparently destroys the good landscape. Figure 1 (b) shows a photograph of Yasukuni Shrine, Tokyo,
with a high-rise building behind disturbing the landscape. The reasons such destruction of good landscapes have
occurred are (1) the area of landscape preservation was limited to the near surrounding area of historical buildings,
(2) no regulation existed to the outside of the landscape preservation area, (3) municipal bylaws for regulating
landscapes did not have legal power in Japan. Now that the Landscape Act (MLIT 2004) came into force in 2004, to
prevent landscape destruction and to preserve good landscape, regulation of height of buildings and other structures
both in and outside of the landscape preservation area can be enforced. In order to properly set the height regulation,
it is necessary to figure out the maximum height that does not disturb the landscape from the viewpoint fields for
each surrounding place. It is important to note that the terrain elevations of surrounding area vary from place to
place.
In order to evaluate the maximum height that does not disturb the landscape, called invisible depth, for all the points
behind the specific objects making good landscape, the following four methods have been employed in practice: (1)
279
9
th

observe at the site; (2) make a physical model of the area and buildings; (3) interpret aerial photographs; and (4)
make a numerical 3D terrain and building model. The method (1) is easy but not accurate; the method (2) requires
great time and care; the method (3) may be used for the terrain with no buildings and trees; the method (4) can be
used for fixed viewpoints but one cannot evaluate the case when viewpoints are moving.

FIG. 1: (a) Phoenix Hall of Byodoin, Uji, Kyoto. (b) Yasukuni Shrine, Tokyo.
Recently, Virtual Reality (VR) technology is often used for observation and evaluation of landscape by city
planners, designers, engineers, developers, and administrators. VR and 3D urban and natural models allow the user
to explore various landscape scenes from multiple and moving viewpoints. However, if VR is employed in order to
evaluate the invisible height for wide area behind the historical or valuable buildings or structures, one must develop
a detailed and precise 3D city model with existing buildings, trees, and other objects. This could take a long time
and high cost. If such a city model has already been built for other reasons, it can be used without additional cost.
Unless otherwise, making a large 3D VR model may not be a suitable choice just for obtaining the invisible height
in terms of cost-benefit performance.
On the other hand, Augmented Reality (AR) has attracted attention as a technology similar to but different from VR.
AR technology provides a facility to overlap real video images with virtual computer graphics images. This can be
done by showing a special marker to the video camera attached with a head mounted display (HMD) worn by the
user. The marker is linked with a designated object and the system shows the object image on the marker of the
video screen. The similar thing can be done by wearing see-through glasses with a video camera.
The authors perceived that invisible heights from multiple and moving viewpoints can be evaluated using AR
technology without making an expensive 3D VR urban model. Therefore, the objective of this research is to develop
a new methodology for building an invisible height evaluation system to preserve good landscapes using AR
technology.
2. METHODOLOGY
2.1 Overview of the Invisible Height Evaluation Method Using AR
The main idea of the proposed method is when the user observes the landscape object under consideration from the
viewpoint fields, wearing a HMD and a video camera connected to a PC, the AR system displays gridded virtual
vertical scales that show elevations from the ground level and that are located behind the landscape object, on the
HMD with overlapped real video images. The user, then, captures the image and observes the maximum height that
does not disturb the landscape for each virtual vertical scale. This process is iterated for various viewpoints, and
appropriate maximum height for each location behind the landscape object is determined. Then, virtual vertical,
maximum height scale models that should not disturb the landscape are generated and the user confirms whether the
virtual objects are surely invisible, while walking around the viewpoint fields and wearing the AR system.
2.2 Implementation of the Proposed Method
A prototype system was developed for validating the methodology proposed in section 2.1. As for the AR,
ARToolKit (HIT Lab 2009) was used because it is commonly and widely used for AR research in the world. The
280
9
th

authors used a standard spec laptop PC, SONY VGN-SZ94PS with RAM of 2.0 GB, VRAM of 256MB, a 1280x800
display, OS of Microsoft Windows XP. A HMD of eMagin, Z800, 3D Visor and a web camera of Logicool QCam
Pro for Notebooks with 1600x1200 pixels were used. The web camera was attached with the HMD.
A marker of the AR system was made for identifying the location and viewing direction of the user. Markers are
usually small, for example, 50x50 mm, for the use of tabletop or desktop AR. However, as the landscape objects are
buildings in this research, the typical size of the virtual, vertical scale is about 300m, and the distance of the scale
from the viewpoint can be up to 5 km, small markers such as 50x50mm may not be visible from the viewpoints and
the numerical errors due to the small size of the marker can be very large. Thus, a marker of which size is
900x900mm was made, as shown in Figure 2.

FIG. 2: Photograph and drawing of the marker.
Virtual vertical scale was developed as an OpenGL computer graphics (CG) object. The shape of each scale is a
rectangular solid which consists of multiple 5m-depth colored layers. Each layer has different color so that the user
can read the height of the scale. In addition, the scale object must be see-through or very thin. Otherwise the scales
would cover the target buildings and the user could not read the maximum invisible elevation for each scale.
3. DEMONSTRATION EXPERIMENT AND RESULTS
To demonstrate the proposed methodology and the developed prototype system, an experiment was executed. First ,
Convention Center and adjacent Gymnastic Hall of Osaka University were selected as an experimental landscape
preservation target because these buildings have highly evaluated property of aesthetic design and no permission
was necessary to perform the experiment. Then, the horizontally flat and open square in front of the center and the
hall was selected as a viewpoint field as shown in Figure 3. The marker was installed at the square.
Then, 50m grid was drawn on the map of Suita Campus, Osaka University as shown in Figure 4. The horizontal axis
was named alphabetically, i.e., a, b, c, etc., and the vertical axis was named in number order, i.e., 1, 2, 3, etc. Each
grid cell was named according to the horizontal and vertical number, e.g., d12, k16, m9, etc. The highest elevation in
each grid cell was measured on the map and was assumed to represent the elevation of the cell. The virtual vertical
scale of rectangular solid was placed so that its bottom elevation is the same as the ground elevation of the cell. This
can be done by measuring the location, including the elevation, of the marker, computing the elevation difference
281
9
th

for each cell, and linking the marker and all the scale objects. Table 1 shows the elevation difference between the
marker and all the cells. If all the virtual scales are displayed on the screen, the scale would not be visible or at least,
not readable. Thus, for each time, one row is selected and shown on the screen, and then, the next row is selected
and shown, and so forth.

FIG. 3: Convention center and Gymnastic Hall of Osaka University.

FIG. 4: Gridded map of Suita campus, Osaka University.
282
9
th

TABLE 1: Height differences between the location where the marker was placed and grid cells.

FIG. 5: Photographs taken during the experiments at Osaka University.
The experiment was performed by two students (Figure 5). One student wore the HMD and video camera and
looked at the buildings the scales. The other held and operated the AR system and the PC, and captured images.
Sample captured images are shown in Figure 6. From the captured image, the maximum invisible height for each
rectangular solid scale was measured. They also walked around the square and confirmed that it was possible to
view both the real video image and virtual scales, while walking.
Based on the invisible height measured from the captured images, a sample regulation plan was made as shown in
Table 2. Then, all the scales were arranged so that each height was the same as the regulated height and linked to the
marker. The experiment showed that the virtual shortened scales looked shorter than the target buildings from the
viewpoint field (Figure 7).
283
9
th

FIG. 6: Captured images of the real buildings and marker with virtual vertical scale rectangular solids.
TABLE 2: A sample regulation plan.

FIG. 7: Captured images of the real buildings and marker with virtual scale rectangular solids of which height are
shortened so that they comply with the proposed regulation plan.
284
9
th

4. DISCUSSION
Since ARToolKit is based on the computer vision technique depending on the image of a physical marker on the
video display, errors are born-nature and inevitable. Calibration of camera images must be done and markers should
be carefully and precisely made and installed. Markers should be displayed in a large size relative to the computer
display because the precision depends on the number of pixels representing each edge of the marker. Thus, the
markers should be large enough and should not be placed far from the video camera. Furthermore, the orientation of
video camera relative to the marker and the distance between the marker and the virtual 3D object are important in
terms of the error and preciseness.
The authors executed an experiment to measure the errors prone to the marker orientation and the distance between
the marker and the virtual object. The marker was set at the distance of 7m from the video camera. Four existing real
buildings which are visible from the experiment site and of which precise location and dimension data can be
obtained were selected. Then, virtual 3D wireframe rectangular solid models representing the edges of those
buildings were made using OpenGL and linked to the marker. The distance between the marker and each building
was 124.1 m, 428.2 m, 964.2 m and 2,851.2 m. The orientation from the marker to the video camera varied 0, 15,
30, 45, 60 degrees. A photograph of the site for the case of 2,851.2 m is shown in Figure 8. For each case, the error
of each node of the angle between the actual video image of the existing building and the wireframe virtual CG
model located at the building place in terms of the number of pixels. Figure 9 shows the errors of number of pixels
for the farthest building case (distance = 2,851.2 m). For the orientation of 15, 30, 45 degrees, the errors are less
than 5 pixels, while for the 0 and 60 degree cases, the error was over 10 pixels. Figure 10 shows the relationship
between the average height errors in meter, converted from the pixels, and the distance between the marker and the
existing buildings for 5 different orientation cases. Apparently, the cases of 0 degree, i.e., the video camera was
located just in front of the marker, indicated large errors of over 15m, which suggests the inability. However, for
other cases, including the farthest building, the average errors were less than 7m.

FIG. 8: The experimental site for the case of distance = 2,851.2 m. Points A and B are the nodes of the virtual
building model. The building is 50 story high.
285
9
th

FIG. 9: Errors of number of pixels for the farthest building case (distance = 2,851.2 m)

FIG. 10: The relationship between the average height errors in meter, converted from the pixels, and the distance
between the marker and the existing buildings for 5 different orientation cases.
5. CONCLUSION
In order to preserve good landscape, regulation of height of newly designed buildings is necessary. VR technology
and other methods been employed to measure the allowable height for buildings and other structures near the site.
However, those methods have drawbacks. Thus, in this research, the authors proposed a new methodology for
evaluating the invisible height of virtual buildings that may be designed in the future from the multiple viewpoint
fields using AR technology. Then, the prototype system was developed and applied to a sample good landscape site
at Osaka University. To reduce errors, a large marker was built. Experiments for evaluation errors were executed
and error analysis was performed. Based on the maximum invisible height at various locations near the site, a
sample regulation plan was made. The experiments showed the feasibility and practicality of the proposed
methodology. For future work, as the size of the marker is limited, new methods such as using existing large
structures as markers are explored.
6. REFERENCES
HIT Lab (2009). ARTookKit. http://www.hitl.washington.edu/artoolkit/
MLIT (2004). Landscape Act. http://www.mlit.go.jp/crd/townscape/keikan/pdf/landscapeact.pdf

286
9
th

AN EXPERIMENT ON DRIVERS ADAPTABILITY
TO OTHER-HAND TRAFFIC USING A DRIVING SIMULATOR
Koji Makanae, Prof.,
Miyagi University;
koji@makanae-lab.jp http://www.myu.ac.jp/~makanae
Maki Ujiie,
ARRK Corporation
ABSTRACT: Traffic orientation is of two typesleft-hand traffic (LHT) and right-hand traffic (RHT). With the
increasing cross-border movement of people and the development of worldwide motor vehicle traffic network, the
difference in traffic orientation becomes a barrier in the international traffic environment. The objective of this
study is to clarify drivers adaptability to other-hand traffic (right-hand traffic for Japanese drivers) by conducting
an experiment with a driving simulator, which can easily switch the traffic orientation. According to the results of
the experiment, most of the subjects recorded route mistake in the experiment and feel uncomfortable in the other-
hand traffic orientation, and half the subjects recorded route mistakes in the experiment.
KEYWORDS: Right- and Left-hand traffic, Drivers Adoptability, driving simulator, worldwide traffic network

1. INTRODUCTION
Vehicles travel on the left or right side of the road. The Convention on Road Traffic agreed upon in Geneva in 1949
stipulates that each contracting state should adopt an uniform travel system but has not defined any standard
international vehicular travel system. At present, 75 countries and regions adopt the left-hand traffic system (LHT)
and 164 the right-hand traffic system (RHT). Several countries have changed the system, mostly from LHT to RHT.
As a result, RHT is now prevalent.
Drivers learn to drive under a particular traffic system adopted in their country. As more people move across the
national border in recent years, opportunities have been increasing of driving under a different traffic system. The
construction of big civil engineering structures such as strait-crossing bridges and tunnels has created global road
networks. Then, safely connecting networks incorporating different traffic systems is a present issue. The means for
removing the barriers created by the difference of traffic system include globally standardizing the traffic system
and enforcing drives operating a vehicle across the border to learn driving for adaptation to a different traffic system.
In either way, drivers need to adapt themselves to varying traffic systems temporarily or permanently. Against the
above background, the authors developed a driving simulator (DS) applicable to varying traffic systems. In this
study, driver's adaptability to different traffic systems is evaluated by having domestic drivers who have accustomed
themselves to LHT drive under RHT using the DS.

2. SIGNIFICANCE OF TESTS USING A DRIVING SIMULATOR
Studies of driver behavior frequently require testing under special environments or conditions. Tests using actual
vehicles under actual conditions involve risks for drivers or surrounding environments in numerous cases. Applying
a driving simulator (DS) to these tests enables the reproduction of a certain point in time under the same
environment for numerous subjects because it ensures the safety of subjects and because it can reproduce the test
environment and control test conditions. These benefits are the reasons why DS have been used for various tests and
training such as the tests of driving burden and the assessment of road design. In order to evaluate how scared the
driver is, Matsui et al. (2002) verified the relationship between the distance between vehicles and driver's scare in
tests using a DS. Tests for evaluating the fatigue of drivers who travel long hours were conducted by Nishida et
al.(2003) and others. Thus, DS have been employed as a useful tool for conducting tests that would be difficult
under an actual environment because of the need to ensure the safety of subjects.
287
9
th

DS involve high-speed dynamic image processing with the aid of computer graphics that work in cooperation with
the operation of the steering wheel. DS therefore used to be developed based on costly graphics-oriented work
stations. In recent years, however, DS systems can be built based on personal computers owing to the advancement
of image processing capability of personal computers (ex. FORUM8 UC-Win/Road). In the meantime, DS systems
have been developed for practical application that are equipped with a motion controller capable of representing not
only changes in image and sound but also changes in sense of equilibrium due to acceleration or centrifugal force
(Onuki et.al., 2006).

3. DEVELOPMENT OF A TRAFFIC SYSTEM DRIVING SIMULATOR
3.1 Composition of the driving simulator
The driving simulator (DS) discussed in this study is intended for a comparative study of driver adaptability at the
time of switchover to a different traffic system. The requirements specified in DS development are the capabilities
of switchover to a different traffic system, specification of road structures on an ordinary road or expressway
according to the switchover, and control of the timing of emergence of other vehicles. Providing high-resolution
pictures is, however, not so important. In this study, therefore, a personal computer-based dedicated DS is developed
and tests are conducted.
FIG.1 shows the composition of the DS developed in this study. The DS is composed of a personal computer (with
Microsoft WindowsXP), driving control system (Microsoft Sidewinder Feedback Wheel) and liquid crystal display
projector. The driving control system is connected to the personal computer via a USB port.

FIG. 1: The composition of the DS

Languages used for developing the DS are C# as a basis, OpenGL for drawing and DirectX9.0 for acquiring
operation data from the driving system. Course data can be expressed by XML and DOM (Document Object Model)
is used for data input and output.
3.2 Acquisition of driving data and drawing of the vehicle
The data on the operation of the driving system is acquired using DirectInput. Data on the angle of rotation of the
steering wheel and the degree of acceleration or braking is obtained as integers between 0 and 1000. The data is
converted to the angle of rotation of the steering wheel and acceleration or deceleration to calculate the position of
the vehicle and generate an image. A DS user interface is shown in FIG.2. As the initial settings for the DS,
whether or not the position of the steering wheel is changed, whether or not dynamic images of other vehicles exist
and whether or not the degree of control is displayed are specified, and drawing is carried out accordingly.
288
9
th

FIG. 2: The DS user interface
3.3 Acquisition of driving data and drawing of the vehicle
Objects of drawing are the road, accessories to the road, surrounding facilities and oncoming vehicles.
(1) Road
The road object is constructed based on the data on the road alignment composed of straight lines, circular curves
and clothoids; and the road cross section. Based on the object, such objects as crossings, T intersections and
interchanges are also prepared as parts and drawn according to their positions.
(2) Accessories to road
As accessories to the road, traffic signals, road signs and vegetation are specified. Traffic signals are designed to be
controllable according to time. The drawing function is specified so that the positions of traffic signal supports may
be changed whenever the traffic system is switched. Multiple types of road signs are generated including regulatory,
warning and guide signs. The supports are also re-positioned when the traffic system is switched as traffic light
supports. Trees can be arranged as vegetation, which is represented by billboards using two textures.
(3) Surrounding facilities
Multiple types of structures can be arranged along the road. Texture mapping is possible for some structures.
(4) Oncoming vehicles
Oncoming vehicles are represented by texture mapping. They are designed to emerge in relation to the position of
the test vehicle.
3.4 Output of travel data
Vehicular travel data is recorded using XML. The current position (coordinates), angle of rotation of the steering
wheel and the degree of acceleration or braking are recorded and output every 0.5 second.
3.5 Construction of an analysis system
An analysis system is built to analyze the travel data recorded by the DS. Graphs and plan views can be drawn for
the speed, time and tracks based on the travel data of a single or multiple subjects.
3.6 Construction of an analysis system
A course editor is generated to facilitate the arrangement of road objects, accessories and surrounding facilities on
the experimental course (FIG. 3). Data is described using XML.
289
9
th

FIG. 3: The course editor

4. ADAPTABILITY EVALUATION TESTS
4.1 Test method
Tests were conducted for 13 subjects with a driver's license for an ordinary motor vehicle who had never driven
overseas. Ten males and three females of an average age of 22.1 participated in the tests. Test courses were prepared
for LHT and RHT. The course was composed mainly of an ordinary road, expressway and interchanges. The round-
trip course had right- and left-turning curves, merging zones and entries into parking space, which are the key points
when comparing the travel under different traffic systems. Subjects practiced driving for several minutes to get
themselves accustomed to the driving system and travelled under the LHT and then RHT systems. Data such as the
travel speed and tracks was recorded. Questionnaires were distributed to subjects to report the points where they felt
scared. FIG.4 shows the model course map and the points where travel records were compared under LHT and RHT
systems.

FIG. 4: The model course map

290
9
th

4.2 Test results and discussions
FIG. 5 gives graphs showing the time that the subjects required to complete the round-trip. The subjects required an
average of 602 seconds, or approximately ten minutes, to go around the course under LHT and an average of 639
seconds, or approximately 10 minutes and 30 seconds, under RHT, 30 seconds more than under LHT. Actually,
however, less time was required under RHT than under LHT because the course is longer under RHT by as long as
900 m owing to the location of the entrances to interchanges and travelling the distance at a speed of 50 km/hr, the
speed limit specified for ordinary roads, took nearly one minute. FIG. 6 shows changes in speed for subject A.
Subjects travelled at a speed of 71 km/hr under LHT and 75 km/hr under RHT. In response to the questionnaire
distributed after the test, subjects reported that they got accustomed to simulation under RHT because they first
travelled on the left side of the road. The time they spent for practice before the test may have influenced the result.

Upper (red): LHT, Lower (Blue): RHT
FIG. 5: The time to complete the round-trip

Upper (red): LHT, Lower (Blue): RHT
FIG. 6: Changes in speed for subject A
Table 1 gives the number of cases where the subjects explicitly deviated from the lane as indicated by the tracks.
Table 2 shows the points where the subjects felt scared based on the results of the questionnaire survey. The number
of cases of lane deviation under RHT was three to four at intersections (1) and (4). Subjects passed intersection (1),
the first intersection to pass under RHT, while they were not yet accustomed to travelling on the right side of the
road. They therefore may have taken the wrong travel lane (FIG. 7). The cases of lane deviation increased at the
passage of intersection (4) because it was designed that oncoming vehicles would emerge at the intersection.
Subjects probably took the wrong lane while being distracted by the oncoming vehicles. Actually, a vehicle running
291
9
th

on the right lane crashed into another. The subject focused attention on one side and may have been careless about
the right-hand traffic. Intersections scored high in the questionnaire survey as a point where the subjects felt
particularly scared.
Table 1: The number of cases where the subjects deviated from the lane

Table 2: The scared points from the questionnaire survey

(a) LHT (b) RHT
FIG.7 : The trajectories at the intersection(1)

(a) LHT (b) RHT
FIG. 8: The trajectories at the entrance of ramp way
292
9
th

Six subjects out of 14 entered the expressway without noticing the different way of installing the entrance at the
interchange (FIG.8). Under a different traffic system, the merging movements of vehicles on an expressway are
reversed. Subjects may have taken the same path as under LHT. Numerous subjects pointed out in response to the
post-testing questionnaire that particular attention should be paid.
The conclusion is that improper driving frequently occurs at intersections (during left-turning movements in
particular) and at entrances at the interchange when LHT is switched to RHT. These points are therefore considered
risk points. Improper driving is ascribable to distractions by other phenomena in such cases where a traffic system
was switched to another or other vehicles emerged after the passage of an intersection.

5. CLOSING REMARK
In this study, the adaptability to RHT of local drivers accustomed to LHT was evaluated, using a dedicated driving
simulator in order to evaluate the adaptability of drivers to varying traffic systems. Numerous subjects actually
drove improperly under unfamiliar RHT traffic system, running off the road or committing other errors. The post-
testing questionnaire survey revealed that many subjects felt scared under RHT. It was also shown that numerous
driver errors occurred when turning at the intersection or entering the interchange and that the emergence of other
vehicles including those coming toward the subject caused errors.
In this study, analysis was made based on the behavior of a test vehicle reproduced using a driving simulator and the
results of a questionnaire survey. In the future, driver anxiety or sense of fear needs to be evaluated based on more
objective parameters through physiological measurement in such terms as the pulse rate and blood pressure.
Increasing the duration of testing is also necessary to determine the driving experience required for adapting to a
different traffic system. In this study, driving under LHT was simply compared with driving under RHT and a
personal-computer-based simple driving simulation system was developed and applied. In order to increase test
accuracy, it is necessary to conduct tests to evaluate driver adaptability using a driving simulator with motion
control capability that generates a greater sense of reality.
In addition to evaluating driver behavior in testing, studies on social systems are also important. As described in
Section 1, the development of global road networks and the growth of international exchanges are likely to make the
difference in traffic system a barrier in transportation. The two traffic systems established separately in different
countries in the 19th century through the beginning of the 20th century remain, dividing the world. Whether the
switchover of traffic systems and different road networks is possible or not under the present mature transportation
environment is a big research issue in the future. Large-scale social experiments may be required. We should
energetically make efforts to tackle these issues in the future.

6. REFERENCES
Forum8 (2009). UC-Win/Road, http://www.forum8.com .
Matsui,Y., Kim, J, Hayakawa,S., Suzuki,T., Okuma,S., Tsuchida,N.(2002). Evaluation of Fearfulness of Driver
Using Three-Dimensional Driving Simulator, Proceedings of the Human Interface Sympsium 2002, Sapporo,
Japan, Sep. 1-3, 2002, 131-132. (in Japanese)
Nishida, Y., Shirai, Y., Otsubo, T.(2003). Development of a driving simulator program for evaluation of long-time
drive tiredness., Proc. of Traffic Engineering Meeting(JSTE), Vol.23, 97-100. (in Japanese)
M. Onuki, Y. Suda, Y. Takahashi, H. Komine and K. Matsushita, Study on Improving Angular Acceleration Feel of
the Driving Simulator and its Effects on Drivers Behaviors, International Journal of ITS Research, Vol. 4, No. 1,
(2006), 47-52.

293
9
th

C2B: AUGMENTED REALITY ON THE CONSTRUCTION SITE
Lon van Berlo, M.Sc.,
Netherlands organization for applied scientific research TNO;
leon.vanberlo@tno.nl
Kristian A. Helmholt, M.Sc.,
kristian.helmholt@tno.nl
Wytze Hoekstra, B. Eng.,
wytze.hoekstra@tno.nl
ABSTRACT: This paper describes the development of a system that brings augmented reality at the construction
site. Due to the growing complexity of buildings, the growing number of actors on the construction site, and the
increasing time pressure to construct a building it keeps getting harder to gain a clear overview of the work in
progress. This leads to high costs due to construction failures and even security risks of the building and on the
construction site. To cope with these problems it is necessary to develop a tool that helps construction workers to
rapidly gain inside in the (intended) construction and construction site. The C2B (pronounced see to be) system
combines the real world with the virtual design giving a mixed reality (augmented reality) view at the construction
site. The C2B system combines technology from different industries to help the construction industry. This paper
describes the development of a prototype of this system.
KEYWORDS: virtual, augmented, reality, world, construction.

1. INTRODUCTION
The C2B (pronounced see to be, future artists impression above) prototype system provides augmented reality on
the construction site. It was designed and constructed during a project that ran within one of the research programs
at TNO called Mixed Reality and Collaboration. In this project TNO tried to establish how construction processes
at a construction site could be improved by applying new and innovative technology in the area of mixed reality.
295
9
th

1.1 Need for augmented reality
As we have entered the 21
st
century buildings tend to become much more complex. Through the introduction of
building management systems with electro-mechanical systems buildings seem to be almost changing into complex
organisms. Due to specialization of the construction workers more and a diverse set of actors is walking around on
the construction site. Add to that the increasing time pressure to construct a building and it becomes clear that it
keeps getting more difficult to gain a clear overview of the work in progress for each of the actors involved. This in
turn leads to higher costs due to construction failures. Also security risks of the building and on the construction site
will increase.
To cope with these problems described above it is necessary to support construction workers by providing them with
real-time and accurate information about their ever changing surroundings and the task at hand. We want to do so
without overloading them with information, but helping them navigate through the complex environment they have
to work in. This is were technology for augmented reality (sometimes also known as mixed reality) kicks in. By
adding an extra information layer to the real physical world we can provide the right information in situ at the right
time. There are many different ways of how to augment reality. In this article we assume the use of visual
augmentation. We add extra information into a persons visual view of the world. We see three major categories of
applications using this kind of technology:
in situ experience. When a building is to be constructed it will alter the landscape. To experience this
feeling people could walk around in that landscape and look at a virtually constructed building and see
how it fits in with the surroundings. What is the impact of a new office building, house or bridge in
the landscape? Early in a project an architect would want to show his views and ideas to other people.
Also, by using environmental models it could be possible to add more abstract information into the
augmented view, like environmental noise contours surrounding the building. The combination of a
real-life environment with the virtual environment can be very enlightening;
in situ verification. In order to carry out inspections of the construction site a 3D construction map
could be projected on top of a building situation. An inspector could then visually check if the
intended design is in line with how the work was actually carried out. For example the correct
placement of reference poles for brick-laying. This does require a great accuracy; and
in situ warning. When unseen dangers are present, workers can be warned of these dangers in a more
interruptive and attention demanding way than a sign on the wall (with probably outdated
information).
In this project we wanted to find out to what extent we could engineer a system that would support those
applications using on the market components (with some integration technology of our own). To that, we wanted to
get an idea about the level of applicability: is it ready for the construction site, or is it an idea for the (near-by)
future?
1.2 Existing approaches
In order to provide people with a view on an augmented reality, several ways already exist. One of the oldest ways is
to draw a picture of reality as you perceive it and add the extra information. This method is time-consuming and
requires a lot of human labor and a lot of craft when the viewers want a realistic view. This method is rather static
too. Not surprisingly photo montage has entered the stage where artists blend in a building in front of other building.
In the digital age a tool like Photoshop in combination with a CAD system is often used. With enough patience and
skill a good impression can be delivered. However, this is not that immersive. You look at a picture which does not
surround you. In order to get an immersive experience there is the Cave Automatic Virtual Environment (CAVE). It
has been there for more than a decade by now. It is an immersive virtual reality environment where projectors are
directed to three, four, five or six of the walls of a room-sized cube. The use of a CAVE requires that you just as
with the photo montage first capture reality and bring it into to the cave. This is still a rather static experience.
However, you can move around in a CAVE, by telling the system (using some kind of device) that you want to
move around. The CAVE computes what you should see again and projects it on the screen. Although more real,
you are not actually at the construction site. You do not hear it, feel it and smell it. And since you cannot bring a
CAVE to the construction site, the possibilities end here.
296
9
th

Due to the arrival of portable wearable computational and display technology however there is another way of
providing an immersive experience in situ. An important part of this technology is known as the hands-free Helmet
(or sometimes Head) Mounted Display (HMD). It can project a 3D scenery on the retina of the viewer, combining
this with a real-time view of the surroundings. It is a one-person device only and requires a computational device for
computation. Together with todays portable computer power including powerful 3D graphic cards an in situ
immersive image can be created. By combining this with technology to determine location and orientation of the
viewer, the basic ingredients of augmented reality are there. Many different layers of information can be added,
resulting in a better and integral oversight of the construction plan (different aspects ant the same time). This was
already the case six years ago (see Piekarski 2003 and Dias 2003).
2. SOLUTION APPROACH
On current day construction sites a augmented reality outfit like the one described above is not common yet. This
may because of the fact that the technology is less wearable in practice. Also, virtual environements and augmented
reality tend to cause simulator sickness, related to motion sickness (see for example Groet et Bos 2008 ). It could
also because of the fact that four years ago computers were not powerful enough with respect to battery power. In
this article however we will not provide the reader with an in-depth analysis on why this is the case. We will provide
a description on the approaches we took at end of 2008 and in the beginning of 2009 using more recent and higher
precision technology. This enabled us to learn about the barriers that keep augmented reality from helping workers
at the construction site and also how we could remove those barriers. By trying to prototype two versions of an
actual working system, we tried to identify all important issues. The contrast between these two versions helped us
to get an even deeper understanding.
2.1 3D-virtual reality on top of a real-life landscape
Just as in earlier approaches we decided to use 3D virtual reality and combine this
with a view of the real-life landscape, captured by a camera image. We decided not
to use an HMD to start with in order to project this augmented reality, because of
several reasons. A tablet sized PC (dubbed the C2B device) can be used without
having to put on an wearable computing outfit, also it can easily be handed over
especially if in case of a tablet PC - to other people in case of a group experience.
Interaction with such is relatively easy through its tactical interface. Furthermore
tablet PCs are there in many different sizes of scale (e.g. notebook, laptop,
Smartphone, PDA, etc.). And last but not least: the position method in one of the
two approaches required us to add some extra components that would not fit to the
HMD that well. The one drawback is the fact that the hands are occupied. This means that the C2B device is less
useful for construction workers and more useful for inspection workers or architects wanting to share an experience.
In the future, when the very accurate method of determining location and orientation has even more portable
components, we could add them to a HMD also.
2.2 Generic conceptual model
Our generic conceptual process model we use to create an
augmented (or mixed) reality is a continuously looping
workflow containing 6 steps. First we retrieve the location
of the viewpoint in reality and the orientation of the
viewer (a camera) at the viewpoint. Then we retrieve an
image from the viewer. Using the location and orientation
information of the viewer we compute what the
(augmented) view would look like from that position at
that orientation in virtual reality. This view is finally
visually merged and presented to the viewer. Then, we
start at the first step again. In figure 1 there is another step
called retrieve telemetry. This shows were real live
297
9
th

measurement data (e.g. from sensors) could be entered in the virtual reality to create more accurate data.
Due to the enormous development (progression in precision) in localization systems, it is important to be able to
integrate the newest developments available on the market, therefore steps 1,2 are implemented as separated
components that are separated (loosely coupled) from the other components. This gives the flexibility to make an
interface for virtually every localization device, for instance iGPS delivers position and orientation, but differential
GPS for example only position and special sensors are needed for orientation. This interface connects to the main
program with a fixed protocol. Note that the localization and orientation must need to deliver point and vector data.
Information about the location in an area, like the information from an RFID based system like the one used in by
Sanghyung Ahn et al. (Sanghyung Ahn 2008), is not enough for the visual augmented reality as we see them.
Multiple tracking technologies (optical, inertial, GPS) could be used to improve accuracy. The location and
orientation components would then use these multiple technologies and provide a more accurate estimate to the
other components.
This generic conceptual process model was implemented in two ways, to explore different aspects of augmented
reality. Both implementations of the C2B system consisted of a laptop computer, a camera, sensors for determining
location and orientation, and software to integrate the hardware and combine the information as described in the
process model above.
2.3 The semi-fixed high-precision approach
First we took an approach where a high degree of accuracy was quintessential: preferably < 1mm. This kind of
accuracy is required for in situ verification augmented reality applications. The viewpoint view in this case was a
Prosilica-camera placed at the back of a laptop. The image of this camera (1024x768) was not recorded, but
streamed to the laptop display (live). This made the laptop look like a see through laptop (as if there was no screen,
but the user was looking right trough a window).
2.3.1 Retrieving viewpoint location and orientation
The high-precision localization and orientation was realized using
a system called iGPS. This system consists of a set of laser
transmitters and at least one double receiver. Both receivers and
transmitters are wirelessly connected to a base station, which
calculates the position of the camera which is at the back of the
laptop. Due to the fact that the system is based on laser there had to
be a Line of Sight (LoS) between receiver and transmitter. And
since we wanted our users to be able to walk around with the C2B,
the body of the user could block the Line of Sight (LoS). Therefore
we added some redundancy to our prototype. It was equipped with
4 transmitters and 4 double receivers (attached to the laptop). In
theory one double receiver and 1,5 transmitters are enough to
calculate the relative position in the covered area with a precision
of less then 1 mm and an angular precision less than 0.1 degree.
The double receiver is about 10 cm height and the two black bands are the actual laser receivers (360horizontal and
120 vertical receiving angle). We wanted total freedom of movement for the user and the vertical angle was
limiting the operational space. Therefore two double receivers were perpendicular mounted on the laptop enabling
all rotational angles. Due to the earlier mentioned Line of Sight problems, this construction was doubled on both
sides (left and right) of the laptop. Because of this elaborate construction the user can walk around, rotate and tilt the
laptop. The wireless setup gave great flexibility in the setting up the environment and freedom of movement for the
user. Finally the base station calculated the position of the camera (viewpoint location) and the angles at which the
laptop was tilted and turned (viewpoint orientation), based on the data retrieved from the sensors.
According to the system vendor Metris this system could cover an area of 300 x 300 meters (indoor & outdoor),
which is relatively large compared to most other tracking systems. Also multiple users are allowed. This means that
several people could work in the same area with a C2B, all with <1 mm accuracy. At the moment the authors of this
article do not know of any other usable system that has a higher accuracy in a large volume (> 10m x 10m). Note
298
9
th

that if (exterior) walls are built and sensors the transmitters need to be placed inside these walls and (auto-)
recalibrate the system in order to recreate enough Line of Sights.
2.3.2 Computing and mixing the augmented view
In order to superimpose a view of a virtual reality on top of a real camera feed, we first needed to construct that 3D
virtual reality. This was done use OpenSceneGraph (open source library). All of the components are more or less
standards and the big challenge was integrating these into one program. A big advantage of the OSG-library is the
capability of importing files from many different CAD-programs and if there is no direct import possible, there are
many (free and open) convertors available on the internet. The image overlay we used was a CAD-drawing. Using
the OSG-library implies the usage of real-life coordinate systems and since the iGPS system also uses a real-life
coordinate system, hardly any conversions had to be made. Moving 1 meter in real-life is also 1 meter movement in
OSG and vice versa. The above shows that the realization of the software was relative easy because of the use of
standard components. The update rate of the whole system was at least 30 Hz, giving the user enough interaction to
move freely around.
This virtual world was then finally superimposed on the grabbed video image from the camera using the GigE
Vision SDK (a de facto standard in Machine Vision) for video grabbing and presentation. Microsoft Foundation
Classes were used as a basis to combine all components into an application.
2.4 The mobile low-precision approach
For the second approach we assumed there is far less need for precision and more need for mobility and the ability
to get an impression of how a 3D structure would look like in reality. This system would be more suited for the in
situ experience and warning type of applications. The viewpoint in this case was a Hercules Classic Silver
webcam. This is regular camera that can operate with a resolutions ranging from 320x240 to a maximum of 800x600
(with interpolation). With an USB cable it was connected to a laptop. It had a refresh rate of at least 1 image per
second, depending on the speed of the laptop we were using (Dell Latitude D800 or a Dell Tablet XT).
2.4.1 Retrieving viewpoint location
For this approach we wanted to impose as less restrictions on the out-door
environment as possible. So the use of special two-dimensional barcodes like
those used in the Augmented Reality Toolkit (ARToolkit) was forbidden in this
experiment. Using features of well known environments as used in the
MOBVIS project did require an existing environment, so that was neither
suitable. Since we also assumed that we would do out-door use only with a
clean Line of Sight to enough satellites, so we could use a Global Navigation
Satellite System.
We decided we did want to have sub meter accuracy. This ruled out standard
GPS, which can be inaccurate for as much as 10 meters. Also, we needed the
ability to interface with a laptop and keep the entire setup as mobile as possible.
This made us select a Trimble SPS351 DGPS/Beacon Receiver. It makes use
of a technology called differential GPS (dGPS) and uses extra beacons (on
earth) to correct for any possible errors. In our case a mobile phone could be
used for a mobile Internet connection that provided the error correcting signals.
Which is in fact a box with electronical hardware for computational purposes,
with a form factor with dimensions 24 cm length, 12 cm width 5 cm depth.
The receiver is accompanied by an antenna (the GA530), which is mounted on
a pole (about 2 meters high) and is connected to the receiver by cables. The entire setup updated its position at least
once per second. It weighed about 2 to 3 kilograms. Using a Bluetooth connection and an Application Programmers
Interface we were able to transmit the position information to our application on the laptop.
Note that this system does not deliver the same accuracy in all directions. In the horizontal range it is less than 1
meter. In the vertical range it is less than 5 meters (according to the supplier Trimble). This is a significant shift for
actual buildings. Therefore some kind of calibration has to be carried out with a basic reference object in reality that
overlaps with an object in virtual reality.
299
9
th

In this setup the position of the receiver (i.e. the antenna) is (just like in the high-precision experiment) not the exact
same position of the viewer (the camera). Depending on where the camera is mounted on the total setup, corrections
for the actual location need to be carried out. If the camera is mounted on top of or below the antenna, only a small
correction for height is necessary. The moment however the camera is mounted sideways to the pole, the rotation of
the pole (in the horizontal plane keeping the pole vertically all the time) comes into the equation. If the pole is not in
an upright position calculating the actual position of the viewer also requires the angle that the pole is making with
respect to the ground. This would mean including information on the orientation of the pole.
Note that the GPS signal can be sometimes be blocked at a building site. The use of Dead Reckoning Modules
(DRM) based on inertia-based technology would help provide location information during (temporary) loss of the
GPS signal.
2.4.2 Retrieving viewpoint orientation
In order to feed our computer application with real-time orientation information, we decided to use the OS5000-US
solid state tilt compensated three axis digital compass. This is a square (1) piece of electronics with a 0.3 depth
and provides information on the angle of rotation across three axes. There is the compass axis (based on the earth
magnetic field), which provides a fixed reference frame in the horizontal plane. Then there are two axis of tilt (under
a 90 degree angle). This provided us with an orientation vector anywhere on the globe. Note that irregularities in the
earth magnetic field influence this device, as well as other magnetic sources. The device has about a 1 degree
accuracy (in a 360 degree system) given a maximum 30 tilt, which seemed tolerable for buildings that were in
relatively close range.
At a distance of 1 meter an inaccuracy of 1 degree in a certain direction means a deviation of the focal point of the
viewer of 2 centimeters in that direction. At 100 meters a deviation of 1,75 meters can occur. This means that a
building in virtual reality - which is at 100 meters from the viewer - might seem to float in the air when projected on
top of a live camera feed, assuming that the position of the viewer is measured exactly. When comparing this to the
inaccuracy of the location determination (1 meter in horizontal plane, 5 meters in vertical plane), it immediately
follows that the inaccuracy of the orientation is of the same scale somewhere at a distance of several hundred
meters.
This device was connected to the laptop using a virtual COM (serial across USB) connection. It updated the
orientation information at least once every second. By firmly attaching this square device on top of the camera and
have the tilt-axes correctly align with the lens of the camera, we were able to determine the orientation of the
camera. Note that in order retrieve tilt information about the pole, the camera with the OS5000 device had to be
firmly attached in an upright position to the pole also.
Note that deviations in the surrounding magnetic field (e.g. influence of large steel bars) could result in further
errors since the digital compass is based on magnetic sensors. We did not examine this, but suspect this effect is
noticeable in the presence of structures that impact the earth magnetic field. We also suspect that a (digital)
gyroscopic compass / tilt sensor would probably deliver better results. When such a device would become available
to use in the same form-factor as the OS5000, we would be interested in comparing them. .
2.4.3 Computing and mixing the augmented view
Construction construct of the virtual reality was done using the building blocks of Microsoft Windows Presentation
Foundation (WPF) on the Microsoft .NET platform version 3.5. It provides developers with a unified programming
model for building so called rich Windows smart client user experiences that incorporate UI, media, and
documents. A discussion of this programming framework is beyond the scope of this article. We only deal with 3D
Graphics, a part of the Graphics and Multimedia section of WPF. The framework offers a component (also known as
an element) called the Viewport3D. It functions as a window - a viewport - into a three-dimensional scene. A 3-D
scene can be projected onto the 2D surface of this element. The way it is projected is determined by the camera
settings of the Viewport3D.
Within a Viewport3D element a programmer can define a camera object. We used the Perspective Camera with
vanishing points as in reality. In order to have a proper alignment between the view in the real camera feed and the
virtual world camera, it was important to have the same Field of View (FoV). This can be defined as the ratio
between the amount of horizon that can be seen at a certain distance. In terms of a person standing in front of a
window: if the persons stands close to the window he can see a lot of the outside scenery. If he stands at a greater
300
9
th

distance, he can see far less of the scenery outside. Depending on what camera (and of course the lens) one uses, the
FoV changes. Using a calibration board we measured out the distances and determined the FoV of our camera. By
setting set the FieldOfView attribute of the camera object in our application, the FoVs of the real (Hercules) and
virtual (WPF 3D Perspective) camera were aligned.
We then tried to align the origin of the virtual world with a coordinate in the real world. Since we opted for a mobile
approach we had to take into account the effects of being in open wide space. From a theoretical point of view this
immediately caused problems, because of the essential differences of the coordinate systems. The GPS system uses
polar coordinates, assuming a sphere system. The WPF 3D coordinate system assumes a cubical grid. One can
devise mappings, but they always will result in some kind of distortion of reality. Cartographers have know this for
years at least in the 2D case - and have come up with many attempts to remove distortion for certain purposes.
Discussion of this is beyond the scope of this article. Although we do not often perceive curvature of the earth, it is
not something you can neglect that easy. For example if you have a perfectly straight bar of 100 meters and the
ground is completely flat, i.e. it follows the perfect shape of a sphere, the bar will not fit to the ground. Instead, if
you balance it in the middle, both ends will stick out 5 cm above the ground. Because of the inaccuracy of the
location determination and the orientation, we decided this inaccuracy was tolerable for this approach. However,
with objects far away at the distant horizon, the inaccuracy will be noticeably much more severe.
FIG. 1: Camera and coordinate system in WPF 3D graphics (from the Microsoft MSDN website)
In practice we aligned the location of the virtual and real world by choosing an anchor point in reality. That point
was defined as point Zero (0,0,0) in the WPF3D coordinate system. We then built a scene in the WPF3D space by
using the 3D drawing primitives (lines, rectangles, etc.). Coordinates in reality were corrected by using a delta
between the real world coordinates and the anchor point. Note that the height of the camera was also corrected by a
default distance since it was fixed on a pole. Aligning orientation was done in a similar way: we assumed that the
real (magnetic) south to north direction always was parallel to the z-axis running from the positive to the negative
end. Based on that, a simple scenery of objects was drawn, were it was assumed that the cubical geometry of the
virtual world also applied in reality. Which of course was not true, but acceptable with in the hundred meter range.
For mapping of measurement units in the physical reality to measurement units in virtual reality we took a short cut
and did some manual calibration by drawing several square squares in virtual reality and align this with square
objects in reality.
Finally, using the Extensible Application Markup Language (XAML) within WPF we were able to tell (declare)
the WPF framework to automatically map Viewport3D element onto another element that showed the camera feed
from the Hercules camera. Only the objects from the scenery were visible, there was no background of the virtual
reality with in the Viewport3D, only transparent pixels, which allowed the background reality camera feed to be
seen. The combination of the two resulted in augmented reality.
3. USAGE EXPERIENCE
During the construction of the two prototypes we carried out several tests and could experience the usage of the
developing C2B prototypes first hand.
301
9
th

3.1 The semi-fixed high-precision approach
We had the following positive user experiences with respect to the iGPS based system:
the user can work with a highly accurate augmented reality in a relative large volume. One of the
applications could be checking the placement of the poles used for brick-laying or in an even
advanced way placement of these poles. Another useful application could be laying tiles in complex
(color) patterns; with the prototype the original drawings show the correct tile in real life;
in our opinion setting up and calibrating the iGPS system was simple and fast compared to many other
large volume tracking devices. It is possible to carry the setup to a construction site, set it up, perform
some calibration and start using the system; and
the usage of the entire system was intuitive and self explanatory. People tend to get the hang of it
rather soon, once they perceive the C2B as a window on an augmented reality.
There were also less positive experiences in usage:
one of the first problems we experienced was Lag-Time: the system did not respond fast enough to
motion. This was introduced by the iGPS system, which had an update rate of 60 Hz, but did not
deliver new location information every 1/60th of a second. It collected a few samples and sent them in
a burst mode. Using new firmware we this problem was solved;
the need for a Line of Sight restricted the movement of users in the beginning. After repositioning the
receivers, this was nearly eliminated;
the system is a bit more bulky, compared to the original artists impressions. The iGPS system is the
main reason, the receivers are not very small and they also need boxes with batteries and wireless
transmitters;
the position of the users head has to be aligned with the axis of the camera, otherwise the real-life
world behind the laptop and the image presented by the camera does not really match and the window
on the world effect disappears. A HMD does not suffer from this problem; and
due to the fact that a Line of Sight is needed, it is less suitable when walls are erected between the
beacons and the receivers (i.e. so much walls that all redundancy is lost).
3.2 The mobile low-precision approach
We had the following positive user experiences with respect to the Differential GPS and tilt-sensor based system:
while staying in a fixed position with the setup a user can get some kind of impression of an virtual
object in its real surroundings;
users do not have to set up beacons in advance, since the GNSS satellites have already been put into
place by other people; and
a user cannot block the line-of-sight between the Differential GPS receiver and himself.
There were also less positive experiences in usage:
a collection of huge buildings or other large objects will probably block the GPS signal;
the tilt-sensor can be disturbed by strong magnetic fields;
the entire setup is mobile, but not light. The pole with receiver, antenna and a laptop can be carried by
one person (as long as everything is taped together securely), but it requires some strength. It also
turned out that the pole is very useful a support for the laptop;
when moving around the augmented view becomes pretty unstable because of all the components
moving and vibrating. Only when the entire setup is put to the ground again and allowed to rest for a
while, a steady image appears again. This seems partially because the tilt-sensor is very sensitive.
302
9
th

Small movements create different readings which alter the view. On a relatively slow laptop this is
clearly visible; and
a 3D virtual object that is behind a real object on the camera will still be projected in front of the
camera image. This pretty much ruins the illusion. The way to solve this is by creating a
representation of the real object in virtual reality and have this object rendered with the texture of the
underlying camera image. We did not carry this out due to time limitations within the project.
4. TOWARDS SELECTION CRITERIA
We state that it would be useful to have selection criteria that predict what approach would be best, depending on
one of the three major categories of application. Based on our experience we think it is too early to come up with
precise selection criteria. We do state several remarks that should bring us towards the selection criteria during
further research. With respect to in situ verification we strongly suspect that the choice for an approach heavily
depends on the level of accuracy a specific building inspector needs. The 10cm error in GPS-position combined with
the 1 degree error in tilt in the mobile low-precision approach can cause deviations of more than 20 centimeters at a
distance of 10 meters. Depending on whether or not this is precise enough, how mobile the inspector wants to be and
how much financial resources are dedicated to inspections, one of the two approaches is the better one. Note that we
did not carry out studies with a HMD. This might cause simulator sickness (see Dodson et al. 2002) and probably is
a criterion of its own.
In situ experience seems to be less demanding than in situ verification, but we think more research should be done
on user experience before selection criteria can be defined. In general we currently cannot provide (precise) numbers
with respect to percentages in deviation between aspects of augmented reality and physical reality. When we showed
the C2B approaches people tended to react differently. While several people reported it as helpful (e.g. it supports
the imagination), other people focused on the difference in augmented and physical reality (e.g. it is not real
enough). Because we did not carry out a scientific study on the experience on a representative population, we can
not further elaborate on this.
Finally for in situ warning, we suspect that mobility can be more important than precision when it comes to warning
people in the building area. Also, the chance of inducing motion sickness is important in this approach, since people
would walk around continuously. Both approaches in this article lean on visual feedback. We think that audio or
tactile feedback might be of better use. Think of a sound or a vibration in the presence of fall-through openings for
example.
5. CONCLUDING REMARKS
Our goal was to learn about the barriers that kept augmented reality from helping workers at the construction site
and how we could remove those barriers. By trying to prototype two versions of the C2B system, we have been able
to identify the important issues. Apart from the obvious need for real accuracy in situ verification and also in situ
impression applications, the main challenges of creating C2B seem to be in:
1. Integrating and calibrating all the different components. Especially in the case of the mobile approach the
user has to walk around with all kinds of components connected by cables. We suspect this acts as a barrier
too. The arrival of consumer smartphones with a GPS and a compass build-in seems very interesting in that
respect. Although the accuracy is not at the dGPS level, the usage experience in some areas of application
seems to be good enough. See for example the Layar application (www.layar.eu).
2. Aligning the Virtual World with the Real World. We suggest creating (software) components that take into
account that the world is not flat. For example: towers at larger distances should be less visible due to a
curvature in the earth. From a certain distance the viewer should not be able to see the base of a very high
tower. In further research we would like to investigate the amount of computing power (and energy) needed
for the correction of complicated scenes.
3. Ease of deployment when high precision is demanded. The semi-fixed system still needs setup-time. Once
the walls go up users need to (auto-)recalibrate the site. A solution might be to use a signaling medium that
is less obstructed by physical objects, like radio. To our current knowledge however higher-precision (<10
303
9
th

cm range) radio location information systems (e.g. those from Ubisense) still require calibration and initial
setup time. Also, we have no experience how (thick) walls influence the accuracy of these systems. We
would welcome experimental data in those areas.
In order to solve the issue of more accuracy in orientation (in the mobile approach), we need a component that is far
less depending on the earth-magnetic field. This component should fit on a wearable camera and should be low on
the usage of power. We are looking forward to collaboration with organizations which are trying to develop
(hardware) modules like these.
Once we have met these challenges in the future, the C2B system can be made even more powerful by including 4D
renderings (i.e. a 3D model and a schedule). Also we could integrate real-time sensor information to increase the so
called situational awareness of construction workers with recent information.
6. REFERENCES
Piekarski, W. and Thomas, B.H. (2003). Interactive Augmented Reality Techniques for Construction at a Distance
of 3D Geometry,
Immersive Project Technology / Eurographics Virtual Environments Conference, Zurich, Switzerland, 22-23
May, http://www.tinmith.net/papers/piekarski-ipt-egve-2003.pdf
Dias, J.M.S., Capo, A.J., Carreras, J. and Galli, R. (2008), A4D: Augmented Reality 4D System for Architecture
and Building Construction
CONVR 2003: 3rd Conference of Construction Applications of Virtual Reality, Virginia Tech, September
24th 26th , http://dmi.uib.es/~ajaume/recerca/CONVR2003.pdf.
Sanghyung Ahn, Johnson, L.M., Do Hyoung Shin, Dunston, P.S., and Martinez, J. (2008) Prevention of
construction accidents with augmented reality,
CONVR 2008: 8th Conference of Construction Applications of Virtual Reality, Kuala Lumpur, Malaysia,
October 20-21
http://www.convr2008.com/index_files/submissions/Training%20&%20Game-1.pdf
Groen, E., Bos, J.E., (2008), Simulator sickness depends on frequency of the simulator motion mismatch: An
observation, Presence: Teleoperators and Virtual Environments, Volume 17 , Issue 6, Pages 584-593 , MIT
Press Cambridge, MA, USA
Dodson, A., Evans, A., Denby, B., Roberts, G.W., Hollands, R., Cooper, S., (2002), Look Beneath the Surface
with Augmented Reality, GPS World,
http://www.gpsworld.com/gps/application-challenge/look-beneath-surface-with-augmented-reality-726
Trimble SPS351 DGPS/Beacon Receiver, http://www.trimble.com/sps351.shtml
Ocean Server 5000 Family http://www.ocean-server.com/download/Compass_OS5000_Family.pdf
Hercules Classic Silver webcam http://www.hercules.com/uk/webcam/bdd/p/22/hercules-classic-silver/
Windows Presentation Foundation, http://windowsclient.net/wpf/white-papers/when-to-adopt-wpf.aspx
MOBVIS, http://cordis.europa.eu/ictresults/index.cfm/section/news/tpl/article/id/90340
iGPS, http://www.metris.com/large_volume_tracking__positioning/basics_of_igps/
ARToolkit, http://www.hitl.washington.edu/artoolkit/
Ubisense, http://www.ubisense.net/pdf/fact-sheets/products/software/Ubisense-Precise-Location-062409-EN.pdf

304
9
th

DEVELOPMENT OF A ROAD TRAFFIC NOISE ESTIMATION SYSTEM
USING VIRTUAL REALITY TECHNOLOGY
Shinji Tajika, Graduate Student,
Department of Civil and Environmental Engineering, Chuo University
shin-t@civil.chuo-u.ac.jp
Kazuo Kashiyama, Professor,
kaz@civil.chuo-u.ac.jp
Masayuki Shimura, General Engineer,
Civil Engineering and Eco-Technology Consultants
shimura@kensetsukankyo.co.jp
ABSTRACT: This paper presents a road traffic noise estimation system using virtual reality (VR) technology. This
system exposes an observer in VR space to the acoustic information of the road traffic noise synchronized with
vehicle CG animation. Road traffic noise is computed based on the geometric acoustics theory. The IPT (Immersive
Projection Technology) is employed in order to create VR space, and CG animation is also created by OpenGL. The
observer can change the road environment and the vehicle condition in VR space by the operation of the controller.
The acoustic information created by the present system is calibrated to correspond with the acoustic information
measured by the sound level meter. The present system is shown to be a useful tool to predict the road traffic noise
in planning and designing stage of road.
KEYWORDS: Road traffic noise, audio-visual stimuli, geometric acoustic theory, immersive projection technology
1. INTRODUCTION
It is important to estimate the noise level for the planning and designing of the road and the sound barrier, because
the road traffic noise may cause stress, disordered sleep and defective hearing to the inhabitants who live adjacent to
the roads. Recently, in order to estimate the road traffic noise, a numerical simulation based on the geometric
acoustic or wave acoustic theories has been developed. Generally, the numerical results are visualized using
computer graphics (CG). However, it is difficult to understand the noise level intuitively, because the visualization is
not a suitable tool to express the acoustic information. The tool that can present the computational result for acoustic
information is necessary for the citizens who are not the acoustic specialists. So we are developing the virtual
experience system about road traffic noise.
In the past studies, several systems that expose road traffic noise as the acoustic information have been presented.
Nagano et al. (1999) developed a system that allows the user to hear noise by manipulating the regeneration levels
of recorded video and sound. Makanae et al. (2004) developed a simulator that allows the user to hear noise in
synchronization with traffic flow simulation. Mourant et al. (2003) developed several traffic noise simulators for
driving simulator. However, there are quite few attempts to develop a system that allows the user to hear noise in
various conditions and to view a vehicle animation of stereoscopic view and real scale like reality.
This paper presents a road traffic noise estimation system using virtual reality (VR) technology. This system
presents the acoustic information of the road traffic noise based on computational results and the visual information
of the vehicle CG animation to observer in VR space. In order to compute the road traffic noise level, the ASJ RTN-
Model 2008 (Research Committee of Road Traffic Noise in the Acoustical Society of Japan : 2009) based on the
geometric acoustic theory is employed. The VR space is created by the IPT (Immersive Projection Technology)
(Wegman et al. : 2002) and the active stereo method is employed for stereoscopic view. The display function of
interface is developed to realize that the observer can change the road environment and the vehicle conditions in VR
space. In order to check the agreement between acoustic information created by the present system and the
computational results, the acoustic information is measured by a sound level meter. The present system is shown to
be a useful tool to predict the road traffic noise in planning and designing stage of road.
305
9
th

2. A ROAD TRAFFIC NOISE ESTIMATION SYSTEM USING VR TECHNOLOGY
Figure 1 shows the concept of the present system. The system provides the vehicle driving animation by the real
scale stereoscopic view in VR space which is realized using the IPT. And, the acoustic information of road traffic
noise based on numerical result which is synchronized with the movement of vehicle is produced. The present
system has following three characteristics. First, the observer can move to arbitrary position and can hear the road
traffic noise that correspond with the position, since the road traffic noise level is computed using the position of
observer in real-time (see (A) in Figure 1). Second, the observer can change the road environment; height of sound
barrier, pavement type and passage years after pavement ((B) in Figure 1). Third, the observer can change the
vehicle conditions; vehicle type, vehicle speed and running distance of vehicle ((C) in Figure 1). Furthermore, the
display function of interface is developed in order to realize the second and third characteristics. The procedure of
creation of VR space is shown in Figure 2.
2.1 Making of vehicle animation
The CG data of the vehicle and sound barrier is created by 3DCG software (3dsMax : Autodesk). The polygons of
CG data are reduced in order to speed up the rendering CG time. The CG data is converted to OpenGL format and
the vehicle driving animation is created by OpenGL. The start of vehicle driving animation is synchronized with the
operation of controller.
VR space
noise
observer
Observer can hear
the road traffic noise.
VR space
Observer can change
the road environment.
VR space
Observer can move to
arbitrary position.
VR space
Observer can change
the vehicle condition.
S
p
e
e
d
u
p
m
o
v
e
m
e
n
t
S
o
u
n
d
b
a
r
r
ie
r
(A) (B) (C)
VR space
noise
observer
Observer can hear
the road traffic noise.
VR space
Observer can change
the road environment.
VR space
Observer can move to
arbitrary position.
VR space
Observer can change
the vehicle condition.
S
p
e
e
d
u
p
m
o
v
e
m
e
n
t
S
o
u
n
d
b
a
r
r
ie
r
(A) (B) (C)
FIG. 1: Concept of present system
Road environment data
Vehicle condition data
Observer s position data
Vehicle s position data
Computation of noise
Making of vehicle
animation
Presentation of road
traffic noise by hearing
information
Creation of VR space
Road environment data
Vehicle condition data
Observer s position data
Vehicle s position data
Computation of noise
Making of vehicle
animation
Presentation of road
traffic noise by hearing
information
Creation of VR space
FIG. 2: Procedure of creation of VR space
VR space
306
9
th

2.2 Create road traffic noise using the acoustic information
In the present system, the road traffic noise level is computed by the ASJ-RTN Model 2008. The change of
frequency by the Doppler effect is also considered. The computation is performed using the position data of the
vehicle and the observer in real-time, and the acoustic information based on numerical results is created by using
MAX/MSP (Cycling '74 ) which is a programming software for music and multi-media.
2.2.1 ASJ RTN-Model 2008
The ASJ RTN-Model 2008 is a model for road traffic noise based on the geometric acoustic theory, which is
developed by the Acoustic Society of Japan. The concept of ASJ RTN-Model 2008 is shown in figure 3. In the
model, the A-weighted sound power level of road vehicle noise is computed and then the sound propagation is
computed. In the sound propagation, two kinds of sounds, a direct sound and a reflected sound, are computed. The
road traffic noise at observers position is evaluated by synthesizing of direct sound and reflection sound.
Computation of A-weighted sound power level of road vehicle noise
The A-weighted sound power level of road vehicle noise
WA
L is evaluated as.
where a is the constant related to the vehicle type, b is the coefficient related to the vehicle speed, V is the vehicle
speed, C is the correction term which is expressed as follows.
where
surf
L ,
grad
L ,
dir
L ,
etc
L are the corrections concerning with the noise reduction with drainage
pavement etc, the change of road vehicle noise by the vertical slope, the directivity of vehicle noise, the rest factors,
respectively. In this paper, the express way is assumed and
surf
L of each vehicle type is given as.
more than 60km/h (vehicle speed):
small vehicle (standard + subcompact size ):
large vehicle (medium + large size ):
less than 60km/h (vehicle speed):
small vehicle (standard + subcompact size):
C V b a L
WA
+ + =
10
log 1
) 1 ( log 4 . 6 log 5 2 . 3
10 10
+ + = y V L
surf
3
) 1 ( log 6 . 3 log 5 0 . 5
10 10
+ + = y V L
surf
4
etc dir grad surf
L L L L C + + + = 2
) 1 ( log 4 . 6 7 . 5
10
+ + = y L
surf
5
Computation of A-weighted sound power level
of road vehicle noise
Observer
(sound receiving point)
Sound barrier Sound barrier
Vehicle
(sound source)
Computation of
sound propagation
(direct sound)
Computation of
sound propagation
(reflection sound)
synthesis of direct sound
and reflection sound
(Computation of A-weighted sound pressure level
at sound receiving points)
Computation of A-weighted sound power level
of road vehicle noise
Observer
Vehicle
(sound source)
Computation of
sound propagation
(direct sound)
Computation of
sound propagation
(reflection sound)
synthesis of direct sound
and reflection sound
(Computation of A-weighted sound pressure level
at sound receiving points)
FIG. 3: ASJ RTN-Model2008
307
9
th

large vehicle (medium + large size):
where y is the passage years of pavement, V is the vehicle speed.
The calculation of sound propagation (direct sound)
The A-weighted sound pressure level
d
A
L of direct sound which is propagated from vehicle is evaluated as:
where
WA
L is the A-weighted sound power level of road vehicle noise, r is the distance in a straight line
between observer and vehicle,
cor
L is the correction concerning with attenuation factors that influences sound
propagation.
Where
cor
L is expressed as follows.
in which
dif
L ,
grnd
L ,
air
L are the corrections concerning with the attenuation caused by diffraction, the
attenuation caused by grand effect, the attenuation caused by atmospheric absorption, respectively.
In this paper, the length of the sound barrier is assumed to be infinity and
dif
L is given as follows.
where
spec
C is the constant related to the pavement type, is a different length between a straight path and a
diffraction path from the observer to the vehicle ( see figure 4 ). In the equation (9), if the sound source can be seen
from the observer directly, the sign of is to be a minus, and if a is larger thanb , ] , min[ b a is to be a .
The calculation of sound propagation (reflection sound)
r
A
L of reflection sound is computed by using mirror image of sound source.
The position of mirror image sound source is assumed to be a position of symmetry of real sound source as shown in
Diffraction path
Straight line path
Vehicle
(sound source)
Observer
Sound barrier
1
spec
C
1 0 <
spec
C
0 <
spec
C
Diffraction path
Straight line path
Vehicle
(sound source)
Observer
1
spec
C
1 0 <
spec
C
0 <
spec
C
FIG.4: Diffraction of sound

cor WA
d
A
L r L L + =
10
log 20 8 7
air grnd dif cor
L L L L + + =
8
0
1 0
1
] ) ( sinh 0 . 17 5 , 0 min[
) ( sinh 0 . 17 5
) ( log 10 20
414 . 0 1
414 . 0 1
10
<
<
+

=
spec
spec
spec
spec
spec
spec
dif
C
C
C
C
C
C
L
9
6
) 1 ( log 6 . 3 9 . 3
10
+ + = y L
surf
308
9
th

figure 5. The A-weighted sound power level of mirror image sound source is computed by equations (1)(6), and
the sound propagation is computed by equations (7)(9).
Synthesis of direct sound and reflection sound
A
L at observers position is evaluated as follows.
where
d
A
L and
r
A
L are the sound pressure level of real sound source and mirror image sound source, respectively.
2.2.2 The Doppler effect
The Change of frequency by the Doppler effect is considered as follows.
here f is frequency of sound source, U is sonic speed,
s
u is velocity of sound source (vehicle), is angle
between straight line from sound source to observer and direction of movement of sound source (see figure 6).
2.2.3 Presentation of road traffic noise by hearing information
The MAX/MSP (Akamatsu et al. : 2006) is employed in order to create the acoustic information of road traffic noise
computed by the ASJ RTN-Model 2008. Three kinds of input data are prepared for MAX/MSP, the road traffic
noise level, frequency and the wave file of vehicle noise. These data is captured by MAX/MSP all the time, for this
reason, the acoustic information can reflect the computational noise level and frequency in real-time.
2.3 Virtual Reality System
2.3.1 Outline of VR system
The IPT (Immersive Projection Technology) is employed for VR technology and the immersive display is employed
for VR display. Figure 7 (a) shows the VR system HoloStage in Chuo University and Figure 7 (b) shows the VR
Real sound
source
Observer Observer
Mirror image
sound source
Reflection sound
Real sound
source
x x
Real sound
source
Observer Observer
Mirror image
sound source
Reflection sound
Real sound
source
Real sound
source
Observer Observer
Mirror image
sound source
Reflection sound
Real sound
source
x x
FIG.5: Mirror image sound source

+ =
10 10
10
10 10 log 10
r
A
d
A
L L
A
L
10
=
cos
s
u U
U
f f 11
FIG.6: Computation of the Doppler effect

s
u
cos
s
u
Sound source
observer
s
u
cos
s
u
Sound source
observer
309
9
th

projector and display system. This system is composed of three large and flat screens and high-performance
projectors corresponding to the screen. The front and side screens are transmissive ones and the bottom screen is
reflective one. The VR space is created by projecting the image on the front and side screens, and the bottom screen
as shown in Figure 7 (b). This system has 7.1ch sound speakers and the VR space is created by the acoustic
information and the visual information.
2.3.2 Computer hardware and network
The HoloStage has a PC cluster system consists of one master-PC and four slave-PCs. The specifications of the PC
cluster are shown in Table 1. The Giga-bit Ethernet is employed for the network of PC cluster. Figure 8 shows the
network configurations. A slave-PC computes the coordinates of location of viewpoint sequentially. Other 3 slave-
PCs create the stereoscopic image from the view point rapidly.
2.3.3 Tracking system
The observers motion is captured by a system called VICON Tracking system, which is the optics type motion
tracking system. The positions of markers fitted to liquid crystal shutter glasses and controller are tracked by the

VacLer-PC
VacHloe HP xw9JOO workcLaLloo
CPU 0ual Core AV0 OpLeroo Lm) 2.4CHz
Vemory 8CB
CrapHlcc Card Nvldla Ouadro FX46OO 2
6lave-PC 4
VacHloe HP xw9JOO workcLaLloo
CPU 0ual Core AV0 OpLeroo Lm) 2.4CHz
Vemory 8CB
CrapHlcc Card Nvldla Ouadro FX46OO 2
TAB.1: Specifications of PC Cluster System

FIG.8: Network Configuration
MasterPC
HUB
SlavePC
SlavePC1
SlavePC2
SlavePC3
Projector1
Projector2
Projector3
Amp. 7.1chSpeaker
Front
screen
VR display
Bottom screen
Side
screen
MasterPC
HUB
SlavePC
SlavePC1
SlavePC2
SlavePC3
Projector1
Projector2
Projector3
Amp. 7.1chSpeaker
Front
screen
VR display
Bottom screen
Side
screen
FIG.7: VR system based on IPT

(a)
mirror
front screen
bottom screen
side screen
projector
VR space
(b)
310
9
th

tracker device. Figure 9 (a) shows a VICON tracker device. The 6 VICONs are provided in the 3D VR space
surrounded by the three screens. Figure 9 (b) and (c) show the liquid crystal shutter glasses and controller used by
observer. In these figures, the white small ball denotes the marker.
2.3.4 Method of stereoscopic view
The binocular parallax is employed for the stereoscopic view. The stereoscopic view is realized in VR space by
creating the image that corresponds to binocular retinal images, and projecting it to screen. In this system, the active
stereo method employed to realize the stereoscopic view. The observer wear the shutter glasses shown in Figure 9
(b), which are synchronized to the computer display through infrared emitters alternating the left and right eye
viewpoints at 120 Hz. The observers brain combines the two views into a 3D stereoscopic image.
2.3.5 7.1ch sound speakers
The Holostage has a 7.1ch sound speaker system (7 speakers and a sub-woofer). Figure 10 shows the layout of the
system. The speakers are connected to the master-PC (see Figure 8) and output the sound which is reproduce by
master-PC. Each speaker outputs the same sound in the present system.
2.4 Interface function
In order to change the road environment and the vehicle condition in the VR space easily, the interface function was
developed using OpenGL. The interface is displayed in the VR space (see Figure 11 (a) and (b)) by operating the
controller which is shown in Figure 9 (c). The interface position is fixed on the front screen as the observer can
watch the interface easily. The observer can change road environment and vehicle conditions as; the height of the
sound barrier (05m), the pavement type (drainage pavement or dense-graded asphalt concrete pavement) and the
passage years after pavement (020years), vehicle type (standard size or semi-compact size or medium size or
large size), vehicle speed (50100km/h). And, these changes for road environment and vehicle conditions are
reflected in the acoustic and visual information in real-time. Also, other interface function was developed in order to
confirm the selected conditions and the A-weighted sound pressure level at observers position (Figure 11 (c)).

front soroon
s1do soroon
bottom soroon
lL spoakor lO spoakor lR spoakor
SR spoakor SL spoakor
BL spoakor BR spoakor
subwoofor
FIG.10: layout of 7.1ch sound speakers
FIG.9: VICONtracker(a), liquid crystal shutter glasses(b) and controller(c)

(a) (b) (c)
311
9
th

3. CREATION OF VR SPACE
3.1 Calibration of acoustic information
In order to verify whether the computational sound pressure level agree with the traffic noise from the stereo
speaker, the output sound was measured by the sound level meter (LA-2560 : ONOSOKKI ). The road environment
and the vehicle conditions selected are shown in Table 2. Figure 12 shows the seen of measuring sound level. The
sound level meter was set up on height of 1.4m. Figure 13 (a) shows the unit pattern of the computational and
measuring results after calibration. The input data of noise level to MAX/MSP was calibrated to match the
computational results to measuring results. From this figure, it can be seen that the computational results are in good
agreement with measuring results.

FIG.11: Interface function

(c) Display of conditions and noise level

(a)Selection of changing items (b) Selection of condition
FIG.12: Measurement of A-weighted sound power level

TAB.2: Road environment and vehicle condition
Height of sound barrier : 0m
Vehicle type : standard car
Vehicle speed : 50km/h
Pavement type : drainage pavement
Passage years after pavement : 5 years
Horizontal distance from traffic
lane to observation point
: 5m
Height of sound barrier : 0m
Vehicle type : standard car
Vehicle speed : 50km/h
Pavement type : drainage pavement
Passage years after pavement : 5 years
Horizontal distance from traffic
lane to observation point
: 5m
312
9
th

3.2 Creation of VR space
Figure 14 shows the situation of hearing the road traffic noise based on computational result in VR space. The
observer can hear the road traffic noise considering the Doppler effect, which synchronized with the movement of
the vehicle from arbitrary position by wearing the liquid crystal shutter glasses and operating the controller. Also,
the observer can share the acoustic information and visual information in VR space with other observers at same
time. Besides, the observer can change the road environment and the vehicle conditions, and can hear the road traffic
noise in accordance with the change. Figure 14 (b) shows the scene changing the height of sound barrier. The
observer can hear the road traffic noise that is decreased by sound barrier. Figure 13 (b) shows the unit pattern of the
computational and measuring results with sound barrier (the height of sound barrier and vehicle speed is assumed to
be 3m and 100km/hr respectively). From the comparative studies in Figure 13, it can be seen that the computed
results are good agreement with the measuring results. From the results, it can be seen that the observer can
understand the noise level accurately by the present system, because the noise level is presented by the acoustic
information.
4. CONCLUSIONS
A road traffic noise estimation system using VR technology has been presented in this paper. The road traffic noise
has been computed based on the geometric acoustics theory and presented by acoustic information in VR space. The
acoustic information has been measured by the noise level meter, and calibrated to correspond with the numerical
result of road traffic noise level. The interface function has been developed in order to change the road environment
and the vehicle conditions in VR space. The key features of the present system are as follows.
FIG.14: A situation of hearing the road traffic noise

(a) (b)
FIG.13: Unit Pattern
(a)
(b)
O
2O
4O
6O
8O
O 2 4 6 8
1lmecec)
A
-
w
e
l
g
H
L
e
d

c
o
u
o
d
p
r
e
c
c
u
r
e

l
e
v
e
l

d
B
)
compuLaLlooal
reculL
meacurlog
reculL
O
2O
4O
6O
8O
O 1 2 J 4
1lmecec)
A
-
w
e
l
g
H
L
e
d

c
o
u
o
d
p
r
e
c
c
u
r
e

l
e
v
e
l

d
B
)
compuLaLlooal
reculL
meacurlog
reculL
313
9
th

As the present system provides the visual and the acoustic information in VR space, the observer can understand
the noise level easily by hearing the noise, and also can share the information with other observers at same time.
The real time simulation is realized by using the controller which can change the vehicle condition and the road
environment in VR space.
From the results obtained in this paper, it can be concluded that the present system provides a useful tool to predict
the road traffic noise in planning and designing stage of road. The verification of the applicability of the present
system to the more complicated road environment is left in the future work.
5. REFERENCES
Nagano T., Yasuda Y., Furihata K. and Yanagisawa T. (1999). "The examination of virtual reality system for noise
experience", The Technical Report of The Proceeding of The IEIC, EA99-56, pp.33-40.
Makane K. and Takahashi K. (2004). " 3D Traffic Noise Simulator for VR Environment." CONVR 2004: 4th
Conference of Construction Applications of Virtual Reality, Lisbon, Portugal, Sept. 14-15, 183-186.
Mourant R.R. and Refsland D. (2003). "Developing a 3D Sound Environment For a Driving Simulator",
Proceedings of the Ninth International Conference on Virtual Systems and Multimedia, Montreal, Canada,
October, pp. 711-719.
Research Committee of Road Traffic Noise in the Acoustical Society of Japan. (2009). " Road traffic noise
prediction model ASJ RTN-Model 2008 proposed by the Acoustical Society of Japan", Jounal of Japan
Acoustical Society of Japan, Volume 65, pp. 179-232.
Wegman E.J. and Symanzik J. (2002). " Immersive Projection Technology for Visual Data Mining.", Jounal of
Computational & Graphical Statistics, Volume 11, Number1, March 1, pp. 163-188.
Akamatsu M. and Sakonda S.(2006). 2061 : A Max Odyssey, Rittor Music.

314
9
th

APPLICATION OF VR TECHNIQUE TO PRE- AND POST-PROCESSING FOR WIND
FLOW SIMULATION IN URBAN AREA
Kazuo Kashiyama, Professor,
kaz@civil.chuo-u.ac.jp
Tomosato Takada, Graduate student,
tomosato@civil.chuo-u.ac.jp
Tasuku Yamazaki, Graduate student,
Department of Civil and Environmental Engineering, Chou University
tasuku@civil.chuo-u.ac.jp
Akira Kageyama, Professor,
Department of Computer Science and Systems Engineering, Kobe University
kage@cs.kobe-u.ac.jp
Nobuaki Ohno, Scientist,
Earth Simulator Center, Japan Agency for Marine-Earth Science and Technology
n_ohno@jamstec.go.jp
Hideo Miyachi, General Manager,
Visualization Division, KGT Inc.
miyachi@kgt.co.jp
ABSTRACT: This paper investigates the application of the virtual reality technique to pre- and post processing for
wind flow in urban area. The accurate shape model for landform and buildings is prepared by the integration of
several GIS and CAD data and an automatic mesh generation method is developed. The visualization method based
on VR technique is employed for both pre- and post-processing to understand the quality of mesh and flow field. The
present system is shown to be a useful tool to investigate the wind flow in urban area.
KEYWORDS: Flow visualization, virtual reality, wind flow, GIS/CAD, VFIVE
1. INTRODUCTION
Numerical simulation is becoming more popular to evaluate the wind flow in urban area in accordance with the
development of hard- and software of computers. However, the following problems are pointed out in practical
computations of this type of problems as: 1) it is difficult to check the quality of shape model and mesh for the
complicated spatial domain; 2) it is difficult to understand the three dimensional structure of flow field since the
computational results normally express on the screen or display.
In order to overcome those problems, this paper investigates the application of virtual reality technique to pre- and
post processing for wind flow in urban area. The accurate shape model for landform and buildings is prepared by the
integration of several GIS and CAD data and an automatic mesh generation method based on Delaunay method is
developed. The visualization method based on VR technique is employed for pre and post processing to understand
the three dimensional structure of mesh and flow field. The attempt to combine the different CG images created by
different visualization software is also performed.
The present system is applied to the simulation of wind flow in urban area in Tokyo and is shown to a be useful tool
to investigate the wind flow in urban area at the stage of planning and designing.
315
9
th

2. VR ENVIRONMENTS
The visualization based on virtual reality (VR) is employed topre- and post-processing for wind flow simulation in
urban area. Fig. 1 shows the exterior view of the VR system HoloStage based on IPT (Immersive Projection
Technology). This system consists of a PC cluster (1 master-PC and 4 slave-PC), 3 projectors, VR display (3 large
screens) and a position (head) tracking system (see Fig. 2). The stereoscopic image from the arbitrary viewpoint of
observer is displayed in VR space (CAVE room) by the position tracking system.The details of the VR system are
described in the reference (Takada and Kashiyama, 2008).
The binocular parallax is employed for the stereoscopic view. The stereoscopic view is realized in VR space by
creating the image that corresponds to binocular retinal images, and projecting it to screen. The active stereo method
employed for the method of stereoscopic view.

FIG.1: VR system HoloStage

FIG.2: Hardware configuration

3. PRE-PROCESSING USING VR TEQUNIQUE
In order to obtain the accurate numerical solution, it is important to create an accurate shape model and a mesh with
good quality. The modeling method using GIS/CAD data is developed. The present method is applied to the
Nihonbashi-area, Tokyo.
In case of the modeling of urban area, the building data is needed in addition with the elevation data. For the data for
buildings, the GIS data (Mapple 2500) obtained by the aerial-photo and -laser surveying are employed. For the data
of land elevation, the digital elevation map issued by the Japanese geographical survey institute is employed. Fig.3
shows the configuration of numerical wind tunnel. The studied urban area is modeled on the circle area. The
arbitrary wind direction is treated by the rotation of the circle area. Fig.4 shows the 3D shape model obtained by
GIS/CAD data. The vertical shape for low-storied buildings is assumed to be straight and that for the high-storied
buildings are prepared by the CAD system. In this figure, the color buildings indicate the buildings prepared by the
316
9
th

CAD system. Fig. 5 shows the surface mesh based on the triangular element. Fig.6 shows the finite element mesh
based on the tetrahedral element. The automatic mesh generation method based on Delaunay method (Taniguchi and
Yamashita, 2001, Takada and Kashiyama, 2008) is employed. The fine mesh is employed near the ground and
buildings. The total number of nodes and elements were 2,458,388 and 14,115,104 respectively.
In order to find the bent element, the evaluation of the quality of mesh idealization is performed by using the
criterion of mesh quality (Freitag and Knupp, 2002). Fig. 7 shows the distribution of mesh quality. The blue collar
denotes the good quality and the red collar denotes the bad quality. Fig.8 shows the seen verifying the quality of
mesh idealization. The observer can see the image in detail from the arbitrary view pointand can share the image in
VR space with other observers at same time.

FIG.3: Configuration of numerical wind tunnel

FIG.4: 3D shape model

FIG. 5: Surface mesh
317
9
th

FIG.6: Finite element mesh

FIG.7: Distribution of mesh quality

FIG.8: The seen verifying the mesh quality

4. WIND FLOW SIMULATION
Assuming the viscous incompressible flow in Newtonian fluids, the wind flow can be described by the Navier-
Stokes equation based on Boussinesq approximation.

(1)
(2)
0 ) (
1
, , , ,
= + + +
ij j jj i i j i j i
u u p u u u
0 =
i , i
u
318
9
th

where
i
u is the velocity, p is the pressure, is the density, is the viscosity of fluid, respectively.
For the boundary conditions, the following boundary conditions are employed.

i i
u u =
i i j j i ij j
h u u p n = + + )) ( (
, ,

where
ij
is the Kroneckers delta,
j
n is the outward unit normal to the boundary.
For the discretization in space, the stabilized finite element method based on the SUPG/PSPG(Tezduyaer 1991,
Kashiyama et al. 2005)
is employed for the spatial discretization of Navier-Stokes equation. For the discretization in
space, the linear interpolation for both velocity and pressure based on tetrahedron element (P1/P1 finite element) is
employed. On the other hand, the Crank-Nicolson scheme is employed for the discretization in time. To solve the
simultaneous equation, the element-by-element Bi-CGSTAB2 method is applied.
A parallel computational scheme based on the domain decomposition method (Kashiyama et al. 2005) is employed
in order to reduce the CPU time and computer storage required. A parallel implementation using the MPI suitable
for unstructured grid is designed for the use on PC cluster parallel computer. For each sub-domain, the processor
associated with that sub-domain carries out computations independently.The details of the discretization and
parallel computation have been shown in the reference (Kashiyama et al. 2005).
The simulation method is applied to the wind flow simulation in urban area, Tokyo, Japan. The no-slip boundary
condition is applied to the surface of landform and buildings. The slip boundary condition is employed on the side
wall. The computed velocity and pressure at nodal points are stored in the data file for flow visualization at every 1
second. Fig. 7 shows the computed streamline which is visualized by the commercial visualization software AVS
(Application Visualization Software). From this figure, it can be seen that the wind flow passed large buildings
shows the complicated flow field.

FIG.9: Computed stream-line

5. POST-PROCESSING USING VR TEQUNIQUE
For the visualization of numerical results based on 3D stereo immersion, the commercial visualization software
AVS Express MPE is employed to know the macroscopic flow phenomena (see Fig.9). On the other hand, in order
to know the microscopic flow phenomena, the open software VFIVE (Vector Field Interactive Visualization
Environment; Kageyama et al., 2000, Ohno and Kageyama, 2007), which is a general purpose visualization software
for the CAVE and is written by VR (Open GL and CAVE library) programming, is employed. VFIVE interactively
shows following visualization objects in the VR space; field line, particle tracer, vector arrows, iso-surfaces, etc. The
major visualization methods of VFIVE are listed in Table 1. The user can choose the visualization method by using
(4)
(3) on
g

on
h

319
9
th

a small portable controller called wand in VR space (see Fig.10). Fig. 11 shows the seen verifying the three
dimensional flow field. The observer can understand the microscopic flow field interactively by using VFIVE.

TAB. 1: Major visualization methods of VFIVE

FIG.10: Menu of visualization method in VR space (for velocity field)

FIG.11: Visualization by stream lines and vector arrows
320
9
th

FIG.12: Visualization process of Fusion VR

The attempt to combine the different CG images created by different visualization software is performed. Fig.12
shows the visualization process of the software, Fusion VR (Miyachi et al. 2005). This software realize that the CG
images created by different software are captured and the composite image is produced in Master-PC, and the
information of composite image is transferred to each Slave-PC. Fig. 13 shows the image of composition of the CG
images created by AVS Express MPE (stream lines) and the VFIVE (vector arrows). Using this type of technology,
the observer can understand both macroscopic and microscopic flow phenomena easily at same time.

FIG.13: Visualization of macro- and microscopic velocity field
321
9
th

6. CONCLUSIONS
The application of the virtual reality technique to pre- and post processing for wind flow in urban area has been
investigated in this paper. An accurate modeling method using CAD and GIS data and an automatic mesh generation
method have been developed. The visualization method based on VR technique has been employed for both pre- and
post-processing to understand the quality of mesh and flow field. The attempt to combine the different CG images
created by different visualization software has also been performed. From the application in this paper, it can be
concluded that the present method using VR technique is shown to be a useful tool to investigate the wind flow in
urban area at the stage of planning and designing.

7. REFERENCES
Takada, T.and Kashiyama, K. (2008), Development of an accurate urban modeling system using CAD/GIS data
for atmosphere environmental simulation, Tsinghaua Science and Technology, Vol.13, No.S1 (Proc.
ICCCBE-XII&INCITE 200), pp.412-417.
Freitag, L.E. and Knupp, P.M. (2002), Tetrahedral mesh improvement via optimization of the element
conditionnumber, Int. J. Numer. Meth. Eng., Vol.53, pp.1377-1391.
Tezduyar, T.E.(1991), Stabilized finite element formulations for incompressible flow computations, Advances in
Applied Mechanics, Vol.28, pp.1-44.
Kashiyama, K., Shimizu, T., Kurahashi, T. and Hamada, H.(2005). Parallel finite element simulation for
orographic wind flow and rainfall using GIS data, Int. J. Numer. Methods in Fluids, Vol.47, pp.575-589.
AVS Express MPE: http://www.kgt.co.jp/english/products/mpe/index.html
Kageyama, A., Tamura, Y. and Sato, T. (2000), Visualization of vector field by virtual reality, Progress of
Theoretical Physics Supplement, No.138, pp.665-673.
Ohno, N. and Kageyama, A. (2007), Scientific Visualization of Geophysical Simulation Data by the CAVE VR
System with Volume Rendering, Phys. Earth Planet. Interiors, Vol.163, pp.305-311.
Miyachi, H., Oshima, M., Ohyoshi, Y., Matsuo, T., Tanimae, T. and Oshima, N. (2005), Visualization PSE for
multi-physics analysis by using OpenGL API fusion technique, Proceedings of the First International
Conference on e-Science and Grid, pp.530-535.

322
9
th

CONSTRUCTION PROCESS SIMULATION BASED ON SIGNIFICANT
DAY-TO-DAY DATA
Hans-Joachim Bargstdt, Prof. Dr.-Ing. M.Sc.,
Bauhaus-University Weimar Germany;
hans-joachim.bargstaedt@uni-weimar.de, www.uni-weimar.de
Karin Ailland, Dipl.-Ing.,
Bauhaus-University Weimar Germany;
karin.ailland@uni-weimar.de, www.uni-weimar.de
ABSTRACT: Everyday life on construction sites is commonly characterized by enormous pressure due to time and costs as well
as difficult logistical requirements. Information Technologies offer great potential to solve these problems. These
technologies are to be used more intensively to find new creative and efficient methods of construction.
Modern simulation tools can be applied with increasing success. Until this point, these tools for the optimization of
construction and logistic processes have predominantly been used only in the start-up phase of a project. However,
projects are often affected by unscheduled constraints and limitations that give reason to deviate from the formerly
optimized plan and to find ad-hoc solutions, especially in the erection phase.
In order to meet these requirements, the application of simulation tools from the planning phase can be extended to
the erection phase. However, a more specific database is needed for this.
The topic of this paper has been developed in response to the question of what the capabilities of using simulation
tools are, and how they might be able to incorporate the available information about the actual status of
construction to improve progress tracking and also enhance dynamic scheduling.
This paper outlines an approach based on accurate day-to-day data for the current project state at any time. These
data then facilitate the simulation of possible variations for ongoing optimization. Thus, the critical path monitoring
and the flexibility in case of changes will be improved immensely. Long-term consequences will be recognized at an
earlier stage.
The basis for accurate data pooling is an efficient and automated survey. This yields two challenges. First, it is
necessary to determine which choice of data is significant and actually needed for evaluating the day-to-day status
in construction progress. Secondly, the required data must be captured as efficiently as possible during the ongoing
working activities.
KEYWORDS: Simulation, optimization, process monitoring, evaluation techniques, day-to-day data
1. INTRODUCTION
Non-stationary construction processes are commonly characterized by a great number of failures and changing
boundary conditions as well as enormous time and cost pressure. In order to ensure the efficient use of valuable
resources in spite of these challenges, a scheduling technique is needed that allows for active control and steering.
Planning methods that feature adequate adaptability and support the description of parallel processes, unexpected
faults, and stochastic and fuzzy parameters are therefore necessary (Hohmann, 1997).
Thus modern simulation tools can be applied with increasing success. Until this point, these tools for the
optimization of construction and logistical processes are predominantly used in the start-up phase of a project.
However, projects are often affected by unscheduled constraints and limitations, especially in the erection phase,
that give reason to deviate from the formerly optimized plan and to find better current solutions.
The concept presented in this paper was developed with particular attention to time control. Experience in this field
demonstrates that bar diagrams are still the predominant tools on most construction sites. Even critical path methods
323
9
th

are seldom used. Simulations, however, can provide much more information, such as the machines productivity,
possible hidden capacities, and reasons for reduced output (Hohmann, 1997).
In order to meet the aforementioned requirements of daily construction business, the application of simulation tools
could be extended from the planning phase to the erection phase. A more specific and dynamic data base is needed
to effectively address this challenge.
2. SIMULATION
Most research approaches that deal with process modelling in construction management strive for the support of
decision-making processes. Therefore, linear and nonlinear optimization models that implicate relevant influences
and boundary conditions are used,. In addition to these, simulation tools are used more and more intensively for
decision-making. Even if the results are not optimal, they provide appropriate solutions for complex problems. The
use of simulation tools becomes increasingly attractive as processes become more complex with multiple influences
that must be considered. In simulation the mathematical complexity can be kept at a lower level compared to the
mathematical linear and nonlinear optimization (Tecnomatix, 2006).
Other branches of industry such as automobile (Bayer, 2003) or shipbuilding (Beiert et al, 2007) already use
simulation with great success. Examples include the planning of assembly lines and logistical processes.
Methods used in these related areas must now be transferred from stationary industries to the building industry. In
comparison the building industry is distinguished by individual products that vary in construction type, size,
function, material and many more attributes. Moreover the construction conditions change with every new project as
well as within the execution phases of the project. And within a construction site the working places change and
move as the construction process progresses.
The authors use event-based simulation, as in Tecnomatix plant simulation (Tecnomatix, 2006), which is already
successfully utilized in stationary industry.
2.1 Strategies
An important challenge in simulation is how to keep the vast amount of possible solutions and variations to a
manageable number. Often the construction sequence is not clearly determined by technological constraints, but can
be varied to a high degree. This degree of freedom is synonymous with a lack of clearly defined objectives in a
project. The better the objectives for a project are defined, the more easily the final binding conditions can be
derived. Specifying parameters can be introduced by the investor as well as by the design engineers, construction
companies, or any other parties involved in the whole project.
Concerning the example given in this paper, the vast possibilities of influencing parameters by parties other than the
construction company shall not be considered. In our focus on the construction company, we also assume that the
construction project is not exposed to any initial limiting conditions. Thus a high degree of freedom is given in terms
of how to manage the construction process and how to choose feasible sequences.
In terms of simulation tools, this freedom makes it possible to search not only for an optimum given by past
experience, but also to include possible other and new sequences. At this point experienced site managers might
argue, that, following this approach, most of the randomly generated sequences and dependencies would need to be
checked by hand to determine whether they can be realized in reality and how they can be valued in terms of time
and resources.
One approach to better find appropriate solutions is the definition of certain construction strategies. They are
derived from past experience, but then described in a generalized manner. They can be used to check on the
improvement of solutions from simulation.
Two alternatives for defining construction strategies are possible. The first alternative is the standardization of
proven patterns from former construction projects. This can be considered as a tool kit, including standardized and
detailed processes, which are then included as subroutines within a bigger set of simulation frames. This works like
a library of simulation subroutines with sets of parameters in order to fit them into the specific simulation model.
324
9
th

The second alternative is the definition of main features of construction strategies. This aims to use simple and
transparent guiding assumptions in order to enforce certain clear sequences within a complex project. Since they are
also determined through experience, they are easy to understand and can also be easily checked.
Even complex construction processes yield many possible basic strategies. Starting with the construction sequence,
a certain pattern (from left to right, from bottom to top, along a certain axis, etc.) can be defined, which then
governs all of the following work steps. Other strategic base lines can be from coarse to fine, first inner, then
outer structures, or destruction before construction.
Other conditions are more likely to be characterized as arguments for favorable priorities. This makes it possible to
distinguish between activities of short and long duration. It enables differentiation between dirty and clean activities,
between wet and dry processes, etc. Concerning the analysis of sequences on real construction sites, often one or
two of these patterns are the main characteristics of the favorable solution (Bargstdt, 2008).
For the successful implementation and reasonable choice of possible execution strategies, however, it is necessary to
appropriately identify the actual building progress. The progress should be described as exactly as necessary and as
efficiently as possible.
3. DESCRIPTION OF THE ON-SITE STATUS
The answer to the question Which data do we really need? is going to be a little more complex. The first aspect to
consider in this context is time control. We need reliable information about degrees of completion and the times of
erection - more detailed information about the processes and sub-processes of the project.
There are different ways to describe the building progress. For example, many research projects draw conclusions
from geometry, by analysing image data or scatter plots. Another source of data is the financial progress. For
example, the currently used reporting system is predominantly made for capturing all information needed for
controlling costs.
With regard to time control, a method will be developed that accurately represents the daily building progress by
identifying single-process steps. This method will be sufficient for time control and more cost-efficient than
methods that capture geometric data, because there is less need for expensive technologies, and the analysis will be
less time consuming.
This method starts from a process model that represents all process steps, dependencies, and supporting documents,
such as delivery notes, site reports, quality checks and acceptance certificates.
The curb of a bridge has been chosen as an example because it represents an almost independent, but typical self-
contained element of reinforced concrete construction. Therefore, the findings are likely to be transferable to other
construction processes, since reinforced concrete is a widely used building material.
3.1 Building process
The process of constructing bridge curbs is not thoroughly described in the literature. Therefore the building process
of several bridges was monitored, captured in detail, and
documented. Different road bridges with varying degrees of
complexity and different constraints were selected. In the
following section, one of these projects is described in detail.
This largest of all monitored bridges is a 445 m long highway
bridge in Germany with 12 spans and a double-webbed t-beam
cross-section made of pre-stressed concrete. This bridge was
built with mobile scaffolding. The formwork for the curb was on
the ground bridge slab (Figure 1).
This bridge shows multifaceted boundary conditions that
strongly influence the building process and methods as well as
the time scheduling. The bridge spans two farm tracks, an
interstate highway, a highly frequented main railroad line, and a

FIG. 1: Bridge curb (Bundesministerium, 2004)

325
9
th

river. Thus all disturbances, including blocking the railroad track, had to be kept to a minimum. The river is part of a
drinking water protection area and is subject to strict environmental regulations.
Right from the beginning of the execution the construction sequence deviated and had to be constantly rescheduled
by the site manager. The ongoing direction of the building progress was based solely on the site managers
competence. He had to act on a day-to-day basis. A thorough consideration of long-term consequences was almost
impossible.
Within the monitoring of the real process, all interferences and the following deviations as well as the corrective
activities were documented in order to use them for the presented approach as well as for future simulation
scenarios. Furthermore, the duration of all activities, as well as the exact need for material, machines, and manpower
was monitored.
Other bridges were also monitored in this way in order to gather a reliable data source. Based on these data a
universally valid and conventionalized construction process for a bridge curb was designed. Event-driven process
chains (EPC) were used to analyze and visualize all dependencies within the building process, because of their easy
application and transparency (Figure 2).
The construction process is structured in horizontal and vertical directions (Buchhop, 2007). In the horizontal
direction it is structured in terms of main, basic, sub, and manufacturing processes. Furthermore, it is structured in

FIG. 2: Construction process of a bridge curb

326
9
th

terms of functions, events, and logical and causal dependency relationships. Every function starts with an event, for
example a certain activity or a special point in time. These events are identified and described exactly.
The vertical direction defines the degree of detail according to which a process is structured. For example one
structural element can be considered as one piece or with all of its single elements. It is defined by the requirements
of the simulation. Thus the processes are structured as detailed as is reasonable in order to implement building
strategies and to identify realizable corrective actions in case of delays.
After structuring, all documents coming along with the building process were monitored along with their start time
and duration. Furthermore, all documents were attached to process events.
Finally all process times were monitored as durations of activities as well as cycle times for recurrent processes
(minimum, maximum, and average).
3.2 Evaluation of sub-processes
When looking at a detailed construction process, sub-processes and activities are evaluated in terms of their
importance and significance in relation to the aggregated process.
The idea in structuring the building process is to identify grid points. This term is adopted in analogy to analytical
algebra. Using this approach, a curves shape can be defined if a sufficient number of discrete points and the type of
curve are known. In the same way a sufficient amount of reliable process information will be used to define the
actual status of work. This information will be taken as grid points for the indication of the actual state of
construction progress. Grid points are set up as control gates from which signal information can be generated or
taken. They indicate clearly, or at least with sufficient certainty, which events have taken place and which activities
are in progress. Therefore, the kind and density of required information is to be determined.
As already mentioned, each process starts with an event, which can be a certain activity for example. These events
have to be identified and described as grid points. Appropriate grid points are, for example, events required before a
certain procedure can start (material delivery), events assigned to the completion of a procedure (milestone), easy-
to-describe intermediate states (finishing of sealing or acceptance of reinforcement), and placement of certain types
of rebar and installation units. Grid points are comparable to a less stringent type of milestone in time schedules.
The identification of the most significant sub-processes is important. The AHP (Analytic Hierarchy Process) method
is used to rank the processes by the mentioned criteria order to identify key processes, which are of particular
importance for the ongoing optimization. Later these processes are identified by significant events in the
construction process. Sensitivity analysis is implemented to check the plausibility of the criteria and their emphasis
in the AHP.
The specification of criteria and their scales are only briefly mentioned here due to their complexity. Examples of
criteria are kind and number of dependencies to other processes, process duration, required resources, fault liability,
error probability, flexibility of execution order and date with respect to parallel processes, and possible correction
strategies etc.
For the criteria, varying scales with values based on deterministic and stochastic concepts are defined. In order to
determine those values, different methods will be used. For example, the flexibility of execution order and date will
be determined by using the Critical Path Method (CPM), so deterministic values will be calculated. Another
example is the process duration. A realistic description of the duration is very important for an appropriate
simulation, but it depends very much on the boundary conditions. Therefore stochastic methods as fuzzy logic have
to be used. Finally, a sensitivity analysis is implemented to check the plausibility of the criteria and their emphasis in
the AHP. Aside from the identification of significant events, the results will be used for the validation of simulation
strategies as mentioned earlier.
Next, the grid points have to be connected with the real building process by progress data.
4. DATA SOURCES
In discussing the available data sources, an evaluation of the usable instruments must be completed. Up to this point
most research projects develop and analyze data gathered from one type of source, for example RFID. But the
327
9
th

reality in construction varies so much that different techniques of data gathering have to be combined. Thus an
appropriate mix of these techniques will be established within the ongoing research project. Therefore, different
sources of information are identified parallel to the process modeling that in total give significant information about
the current state of the construction (Bargstdt, 2008).
4.1 Documents as information sources
The approach to designing an efficient and realistic controlling tool starts with the identification of documents and
other information sources that are widely used on most construction sites. Documents like construction records,
minutes of meetings, or delivery notes are associated with the everyday life on each construction site. Up to this
point, however, they have rarely been evaluated in terms of capturing accurate information about the building
progress daily. These documents are analyzed with reference to the information they yield with respect to the degree
of completion or material deliveries. Signal information as described above is a matter of particular interest.
Furthermore, the common data format is registered, which in many cases is handwriting. Typical information
includes:
deployment/staff data such as number of workers, qualifications, working hours (start/end) and
completed work
machine data such as machine inventory, working hours/place, and quantities completed
supply data such as shipments
test results
acceptance results
disengaging dates
concreting times (start/end)
site measuring quantities
general output documentation (description of daily output)
weather data such as temperature, hours of rain, and wind speed
To extract this information, the authors started researching the question of which documents are commonly
generated on construction sites. This research was based on the literature and on a nationwide survey addressed to
site mangers who work for construction companies of all sizes.
The site documents can be divided into three groups: documentation on a normative basis, on a contractual basis,
and for internal controlling issues. These documents are evaluated in terms of the information they provide about the
construction progress. This information is assigned to categories as shown in Table 1.
Table 1: Information categories
categories Data type Definition
resources supply data Documentation of material entries.
stuff data Documentation of number and qualification of workers. capacities

production output
machine data Documentation of number and type of machines.
site measuring data Data for billing, documentation of quantities and work
completed.
production output
testing /
disengaging data
Data for technical control, state of completion, and limits of
quality.
production influences Progress documentation Keeping evidence. Documents for possible court cases, for
example as correspondence.
production conditions Weather data Documentation of production conditions.

328
9
th

By defining and emphasizing different criteria, the documents significance is evaluated and made comparable. The
following criteria and objects are defined and emphasized on simple ordinal scales (Table 2). The evaluation is made
according to AHP. In a subsequent step its applicability for digitization and automation is discussed.

Table 2: Criteria for document/information evaluation
Criterion Explanation
significance Answers the general question of whether the information is significant to conclude on the
status of the building progress. Knock-out criterion.
reference What can the information refer to?
To a specific activity, structural element, working section, or to the construction site as a
whole.
degree of completion To which degree of completion can a conclusion be reached from the information
provided?
To 100% completion, to accurate progress status, or to predefined intermediate stages?
gathering rhythm How often is the information taken?
When the event takes place: every day, weekly, monthly, not predictably.
gathering probability Is the information reliable?
content type Does the document contain factual or explanatory information?
conclusion type Is it possible to draw a direct conclusion?

As expected, all kinds of testing protocols have a high significance with regard to the identification of construction
progress. Additional staff and machine data can be used for plausibility control, since this information must be
combined with further facts for a reliable conclusion to the construction progress.
Data pooling based on the above-mentioned information sources does not yet describe the building process as
accurately as needed. On the one hand it is handwritten and often biased; on the other hand it is not yet sufficient for
a complete picture of the construction status. Depending on the intended optimization, further information must be
captured. Therefore, more sophisticated instruments for data evaluation are considered. It must also be kept in mind
that additional information sources should not be based on personal observation and should be protected from errors
and bias.
4.2 Instruments for data evaluation
Current technical instruments include RFID (Ergen, 2007), GPS, Photogrammetry, Laser scanning, Tachymetry, and
Bar codes (Chen, 2002).
In all of these fields further improvements will be realized soon. As a result of developments in most technologies it
is possible to push the boundaries that were limiting the applications of these technologies to construction just a few
years ago. An example is RFID technology, where the size of a RFID chip has become one-tenth of its original size
within 5 years, and the price has dropped to 1/100 of the original.
Furthermore, increasingly powerful screening machines create new possibilities for improved application. For ex-
ample the storage capacity of Photogrammetry increased enormously, and it is possible to exploit many more data
points in less time. It even seems realistic that real-time feedback will be possible in the near future (Kutterer, 2006).
In addition to these developments, the exchange and storage of data via the internet has created new possibilities.
Nowadays it is possible to transfer huge amounts of data to any place in the world, to process them in a central
location, and to transfer them back to site. The task is to identify the relevant evaluation instruments and register and
analyze them with regard to their ability to record real-time data. This comprises speed, data volume, precision, and
automation of data. It leads to the question of which instruments are appropriate to recognize parameters relevant for
the building progress. Similar to the document evaluation, constraints and requirements needed for the application of
329
9
th

different instruments are considered. Significant parameters are functionality, type of information, degree of
automation, and type of equipment.
Based on the above examination, the different information sources and instruments are methodically compared to
one another and interpreted with regard to a specific subsequent processing within the system for capturing the real-
time state of construction.
The possible information sources and instruments are evaluated with regard to the construction process to be
controlled. They are systematized and tested on their ability to capture signal information at grid points, both as
single sources and when combined with other types of sources. In addition, a catalogue of typical construction
elements is developed, which will answer the question of how best to capture different kinds of elements.
The instruments are compared based on multiple criteria including reliability (subjective or objective), quality,
flexibility, workability, complexity, comparability with standard values, etc.
5. COMBINING PROCESS MODELLING AND DATA EVALUATION
The basics developed so far concerning process modelling and data evaluation will now be combined into one
model. By using process modelling, the building progress is structured in events, functions, relations, and additional
grid points pursuant to the requirements of deadline monitoring and simulation. Thus, it defines where and when
information should be captured.
Now those points will be matched with the possible data sets as shown in Figure 3. Preference is given to real-time
data. In the first step the model will be fed with data that are taken according to availability and actuality
(information sources). These data are in any case reviewed with regard to their relevance and their ability to describe
the status at a considered grid point. Data with less relevance will be held back for further usability, for example
checking on the plausibility.

FIG. 3: Conception
330
9
th

Thus monitored, the virtual model of the building process is generated in some main elements, but may still be
incomplete. Now further appropriate instruments for data evaluation will be selected and activated in order to
complete the monitored situation. They will supply the monitored model with sufficient information for the
remaining grid points. This supplementary data gathering is taken by means, which are chosen based on the
aforementioned multi-criteria comparison.
Again the question of which instruments will be appropriate to capture the necessary information at certain grid
points has to be answered. At first glance almost all instruments for data gathering have the potential for a large
quantity of accurate data, as long as time and costs dont have to be considered. But in practice neither of these
resources are unlimited. It is also not adequate to extrapolate the further development of the tools to infinity.
For the present time and status a catalogue is established with classifications, that point out which instruments are
appropriate to best describe different significant construction status. Nevertheless the opportunities to increase the
reliability and significance by capturing more data, by using more sophisticated instruments, or by referring to
recent developments in monitoring instruments are also considered.
Using a sample construction process as a test environment, a complete simulation model with different data
evaluation and grid points is developed. On this model, tests are run with regard to data formats, evaluation
intervals, and response times. An important aspect is the fast and integrated data handling, which allows daily
steering. In everyday construction business, this time relation is of particular importance for an accurate daily
simulation. Of course, the automatic generation and evaluation of the data is one special aspect. Another important
aspect is to make use of combinations of data and of redundancies for plausibility checks. At the same time
redundant information may also detect contradictory data, which calls for special algorithms to distinguish between
true and false data. The individual adjustment of the different information sources and instruments is essential for
the efficiency of the whole monitoring system.
6. CONCLUSION/PERSPECTIVES
One objective of this research is to evaluate applications and limits of the presented concept using grid points. A
further aspect is the question of how the evaluated data can be implemented in a simulation tool. Using simulation in
construction processes that have been developed at the Chair of Construction Engineering and Management at
Bauhaus-University Weimar (Beiert et al, 2007), the prototype implementation of different instruments for data
generation will be added to the simulation model. The effects of different performances then will be analyzed.
Simulation models are suitable for developing construction sequences, as shown in (Knig, 2007). This approach
considers the monitoring aspects of construction sites. Data gathering, data processing, and the following steering
processes are implemented in the simulation model. It yields information about the importance of precision,
velocity, automation, redundancy, and significance of data capturing during construction processes. The results,
advantages, disadvantages, weak points, and application limits can be adjusted for different construction conditions.
Furthermore, the error-proneness can be tested. One important aspect of this will be missing data and how the
simulation will react in the case of uncertainties or, for example, a missing delivery note.
With the knowledge gained from these simulation experiments, the results will be verified on a prototype. Even if
the model suggests complete control of the construction process, the reader should be aware that construction is far
too complex to rely simply on automation. The presented approach is a way to improve the information on ongoing
construction sites. It will enable the construction manager to better concentrate on real obstacles that have not been
encountered in advance, and to have better information sources for routine processes to be steered. Thus the time to
react will be shortened. It is then also possible to evaluate effective counteractions by using the simulation tool in
combination with real-time data gathering.
331
9
th

7. REFERENCES
Bargstdt H.-J., Ailland K. (2008), Capturing the real time state of construction for simulation,
IABSE Conference, Helsinki, Finland, Conf. Proceedings.
Bayer J. (2003), Simulation in der Automobilproduktion, Springer, Berlin.
Beiert U., Knig M., Steinhauer D., Bargstdt, H-J. (2007), Constraint-Based Simulation of Outfitting Processes
in Shipbuilding and Civil Engineering, 6th EUROSIM Congress on Modelling and Simulation, Conf.
Proceedings, Ljubljana, Slovenia.
Buchhop E. (2007), Zeitliche Erfassung von Kernprozessen als Teil der Prozessanalyse, bdvb-Award
Geschftsprozess- und Projektmanagement, CT Salzwasser-Verlag, Bremen.
Bundesministerium fr Verkehr, Bau- und Wohnungswesen (2004), Richtzeichnungen fr Ingenieurbauten,
Verkehrsblatt-Verlag, Dortmund.
Chen Z. H. Li, Wong C.T.C. (200x), An application of bar-code system for reducing construction wastes,
Automation in Construction, Vol. 11, no. 5, pp. 521533.
Ergen E., Akinci B., Sacks R. (2007), Tracking and locating components in a precast storage yard utilizing radio
frequency identification technology and GPS, Automation in construction, Vol. 16, pp. 354367,
25.07.2006.
Hohmann G. (1997). Von der Netzplantechnik zur Simulation - Analyse von Bauprozessen mit Hilfe von Petri-
Netzen. Internationales Kolloquium ber Anwendungen der Informatik und Mathematik in Architektur und
Bauwesen , IKM , 14 , 1997 , Bauhaus-Universitt Weimar, Germany.
Knig M., Beiert U., Bargstdt H-J. (2007), Constraint-Based Simulation of Outfitting Processes in Building
Engineering, 24th W78 Conference, Conf. Proceedings pp.491-497, Maribor, Slovenia.
Kutterer H., Hesse C. (2006), High-speed laser scanning for near real-time monitoring of structural deformations,
Tregoning, P., Rizos, R. (Eds.): Dynamic Planet Monitoring and Understanding a Dynamic Planet with
Geodetic and Oceanographic Tools, IAG Symposia, Vol. 130, Springer, 776 - 781.
Tecnomatix (2006). Tecnomatix Plant Simulation, Step-by-step Help, Handbook Version 8.1., Stuttgart.
332
9
th

EFFECTIVENESS OF SIMULATION-BASED OPERATOR TRAINING
John C. Hildreth, Assistant Professor,
The University of North Carolina at Charlotte;
John.Hildreth@UNCC.edu
Michael Stec,
Volvo Construction Equipment North America, Inc.;
Michael.Stec@Volvo.com
ABSTRACT: Construction equipment manufacturers provide on-machine operator training, consisting of individual
instruction regarding general operational performance and machine specific controls. Such training is conducted
in the field using the subject equipment to provide field operational experience. Construction equipment simulators
present an opportunity to train operators for stressful and tough construction environments without needing to
employ an actual machine. Eliminating machine use saves fuel, reduces equipment wear, and lessens the inherent
risks of damage to machine and man.
This study was performed to evaluate the effectiveness of full-motion and zero-motion simulation-based training
using the Volvo Advanced Training Simulator (ATS) relative to on-machine training. Operational performance
before and after training was measured in terms of loading cycle time and production rate. Levels of operator
confidence and anxiety were also collected. Results indicate that full-motion simulation-based training increases
production rate and confidence, while decreasing cycle time and anxiety.
KEYWORDS: Construction Equipment, Simulation, Training, Operator Training.
1. INTRODUCTION
Construction equipment manufacturers are called upon by the industry to provide training for equipment operators.
For Volvo Construction Equipment North America (Volvo), such training is typically on-machine, consisting of
personnel providing individual instruction regarding general operational performance and machine specific controls.
Instructional topics include familiarization with equipment controls, maneuvering the equipment, operational
efficiency, and operational safety. Training is conducted in the field using the subject equipment to provide field
operational experience.
Construction equipment simulators present an opportunity to train operators for stressful and tough construction
environments without needing to employ an actual machine. Eliminating machine use saves fuel, reduces
equipment wear, and lessens the inherent risks of damage to machine and man. Equipment simulation allows
operators to quickly and efficiently be introduced to the equipment and its operation. The benefits of simulation-
based training are perceived to be increased training efficiency and improved operator skills.
Volvo has developed an Advanced Training Simulator (ATS) of an L120F wheel loader. The ATS is a full-motion
simulator designed to aid in training wheel loader operators. The ATS consists of a L120F loader cab mounted on a
hydraulically activated platform with six degrees of freedom, the training simulation software and computer, and an
imaging system to display the three dimensional graphics.
Volvo and The University of North Carolina at Charlotte designed and conducted a research project to evaluate the:
improvements in operator skills realized through simulation-based training;
effect of simulation-based operator training on operator anxiety and confidence levels;
effect of full-motion simulation; and
feasibility of simulation-based operator training.
The purpose of this study was to:
333
9
th

1. aid in understanding proper application of the ATS in developing fundamental skills in operators
possessing no previous loader experience;
2. provide preliminary data regarding improvements realized from simulation-based operator training in terms
of loading cycle times and productivity;
3. provide preliminary data regarding the impact of simulation-based training on operator anxiety and
confidence levels; and
4. provide preliminary data comparing and contrasting full-motion simulation, zero-motion simulation, and
on-machine operator training.
2. BACKGROUND
Simulators are used to provide a safe and inexpensive practice environment for persons who operate complex
machines such as airplanes, trucks, cars, and construction equipment. Simulators are particularly useful to
researchers investigating problems in which the real environment is too hazardous, costly, or difficult to control.
Simulation has been widely applied to the vehicle driving environment to explore the effects of cell phones on
driving performance (Schneider and Kiesler 2005), conditions that lead to better in-vehicle performance (Bullough
and Rea 2001), and devices to help mitigate accidents with in-vehicle warning systems (Enriquez et al. 2001).
Studies have established correlations between participant behavior in driving simulators and behavior in real
vehicles (Godley et al. 2002, Panerai et al. 2001, Tornros 1998).
Simulators are effective training tools because of their ability to replicate a real environment and the realistic
behavior of simulation participants. Two recent studies regarding the use of simulators in training snowplow
operators have been sponsored by state departments of transportation (Strayer et al. 2004, Mashiocchi et al. 2006).
In both reports, simulation-based training was well received by participants and found applicable to operators at all
levels of experience.
Simulation-based training has also been applied to training of construction equipment operators. Wang and Dunston
(2005) present a survey of advanced virtual training technologies for training heavy construction equipment
operators. Gokhale (1995) describes the development of a simulator for training crews to operate the JT2510
directional drilling machine. The objectives of the simulator use were to familiarize operators with the controls,
train the operators to steer the machine, and enable evaluation of operator performance.
The effectiveness of simulation-based training for construction equipment operators is not well documented in the
literature. A multifunctional simulator for training operators of haul trucks, excavators, and surface drill rigs in a
mining application has resulted in improved operator performance and reduced operational damage to equipment
(Anon. 2005). A single study quantitatively documenting the effect of operator skill training was found (Kamezaki
et al. 2008). Participants took part in simulation-based training sessions for double front work machines over a 3
day period. Operator skills in terms of task completion time and positional accuracy were significantly improved.
3. METHODOLOGY
A total of 17 UNC Charlotte students volunteered to participate in the study. Two identical study sessions were
conducted, with 8 students participating in the first session and 9 in the second. The requirements for eligibility to
participate were that students must have no previous experience with operating a wheel loader and be physically
capable of operating a wheel loader. Training sessions were conducted at the Volvo Customer and Demonstration
Center in Asheville, North Carolina and consisted of an information briefing, pre-training testing, operator training,
and post-training testing.
3.1 Informational Briefing
Participants received an approximately 15 minute briefing describing the study and basic wheel loader operation.
Each participant received an information packet containing a description of the study, written pre-test, written post-
test, and wheel loader operating guidelines and tips. During the briefing, participants were randomly and evenly
assigned to each of three operator training methods: full-motion simulation, zero-motion simulation, or on-machine
training.
334
9
th

3.2 Pre-Training Testing
Participants performed an operational pre-test to quantify their baseline operator skills and completed a written pre-
test to document their anxiety and confidence levels. During the operational test, participants operated an L220F
wheel loader to three pass load A35D articulated haulers from a material stockpile. Prior to the start of the
operational test, a loader coach briefed participants regarding the loader controls and truck loading operation.
Each participant was given 3.5 minutes to load each of the three articulated haulers. Because all participants had no
previous wheel loader operating experience, the loader coach was present in the cab during the loading of the first
truck, and subsequent truck loading operations were performed without the presence of the loader coach. The
operational tests were video recorded and the following data collected:
1. truck loading time (sec) from stopwatch timing;
2. number of loader passes; and
3. payload of each truck (tons) as weighed using industrial truck scales.
Immediately following the operational pre-test, each participant completed a written pre-test requiring them to
respond to questions regarding their levels of anxiety and confidence.
3.3 Operational Training
Participants received 25 minutes of operator training in one of three training methods: full-motion simulation, zero-
motion simulation, and on-machine training. Simulation-based training (full-motion and zero-motion) were
completed using the ATS to simulate use of the L120F to load A30E articulated haulers from a material stockpile.
Participants operated the simulated loader while Volvo personnel operated the simulation computer and provided
real-time feedback regarding their performance. In the full-motion simulation, the ATS was operated in the normal
mode with motion capabilities enabled. In the zero-motion simulation, the ATS was operated with the motion
capabilities disabled.
Participants completing the on-machine training used the L120F to load A30E articulate haulers from a material
stockpile at the Volvo Customer and Demonstration Center. The loader coach was present during training to
provide real-time feedback regarding performance.
3.4 Post-Training Testing
Participants completed a written post-test and then performed an operational post-test to quantify their operator
skills. The written post-test required each participant to rate their level of anxiety and confidence in operating a
wheel loader after training. The post-training operational test was conducted in the same manner as the pre-training
operational test. Each participant was given 3.5 minutes to load each of the three articulated haulers. The number of
loader passes for each truck and the loading cycle time based on stopwatch measurements were recorded. Each
truck was weighed using industrial truck scales and the payload of each truck was also recorded.
4. RESULTS AND DISCUSSION
Changes from pre- to post-testing in operational performance were calculated and expressed as a percent change
from the baseline (pre-training) values.
The measured loading time, number of passes, and payload data were used to calculate the production rate (tons/hr)
of the operation. Loading time and production rate data were statistically analyzed to investigate differences
between operator training methods.
Participants responded to questions regarding confidence and anxiety levels as part of the written pre- and post-tests.
Due to the categorical nature, a subjective analysis of the data was performed.
4.1 Loading Time
Loading time was the time required for the participant to load the truck and was measured in seconds using a
stopwatch. Due to the 3.5 minute time constraint for each load, some trucks were loaded with 2 passes and others
with 3 passes. To account for this disparity, a corrected loading time was determined by prorating the recorded
335
9
th

loading time to a 3 pass loading time. The corrected loading time was used for analysis and is presented here as the
loading time. It is clear from Fig. 1 that there was a decrease in loading time after training. Note that FM, OM, and
ZM are used to denote full-motion, zero-motion, and on machine training, respectively.

FIG. 1: Loading Time Box Plot
For each participant, the change in loading time was calculated as a percentage of the pre-training loading time.
This data is provided as Fig. 2, which shows a substantial decrease for all training methods. The greatest decreases
in average loading time were found in full-motion simulation and on-machine training, which were -33.50 percent
and -34.21 percent, respectively.

FIG. 2: Percent Change in Loading Time Box Plot
The percent change in loading time was analyzed for differences between the training methods. Proper analysis
required the data to be tested for normality and equal variances so that appropriate analysis methods could be used to
compare the data. Normality was tested using the Anderson-Darling test. All tests were performed at the 5 percent
336
9
th

confidence level (=0.05). A resulting p-value less than 0.05 would indicate that the data exhibits a non-normal
distribution. A summary of the analyses is presented as Table 1.
TABLE 1: Statistical Summary of Percent Change in Loading Time

Based on the resulting p-values, the hypothesis of normally distributed data was accepted for each dataset. Bartletts
test for equal variance was used since the data was normally distributed. The resulting test statistic was 4.56 and
corresponding p-value was 0.102, indicating that the hypothesis of equal variances be accepted. The normally
distributed data of equal variances was appropriately analyzed using the analysis of variance (ANOVA) technique.
The results of the single factor (training method) ANOVA are summarized in Table 2.
TABLE 2: ANOVA in Percent Change in Loading Time

The ANOVA p-value was substantially greater than the 0.05 confidence level, indicating that the hypothesis of a
common distribution be accepted. This leads to the conclusion that training method does not significantly affect
changes in loading time.
While training method did not significantly influence changes in loading time, it was also important to determine
whether each training method resulted in a significant decrease in loading time. The Students t-test was selected as
the appropriate analysis method based on the previous determination of normally distributed data of equal variance.
The results of one-tail t-tests for decreased loading time are summarized in Table 3.
TABLE 3: t-Test Summary for Decreased Loading Time

The p-values for each of the three t-tests were substantially less than the 0.05 confidence level, indicating that the
hypothesis of no change should be rejected and a significant decrease in loading time be concluded.
4.2 Production Rate
The production rate calculated for this study is the rate at which participants would load material into trucks if trucks
were always available, in position, and loading operations were to occur continuously. In reality, this ideal situation
would not be achieved due to delays in the loading cycle resulting from waiting for trucks to come into position and
inherent inefficiencies in the loading operation. However, the calculated values are relevant given that the scope of
this study is limited to the operation of the wheel loader only. Production rate was calculated as the ratio of truck
payload to loading time.
337
9
th

It is important to note that the loading time used in the calculation is the actual loading time (not corrected loading
time) since the payload values were not also corrected. This calculation resulted in production rates in units of tons
per hour. A box plot of the production rates is provided as Fig. 3 and exhibits the pre- and post-training values for
each training method. From this figure, there is a clear increase in production rate after training.

FIG. 3: Production Rate Box Plot
For each participant, the change in production rate was calculated as a percentage of the pre-training production rate.
This data is provided as Fig. 4, which shows that production rates were substantially increased for all training
methods. The greatest increases in average production rate were found in zero-motion simulation and on-machine
training and were 77.06 percent and 74.37 percent, respectively.

FIG. 4: Percent Change in Production Rate Box Plot
The percent change in production rate was analyzed for differences between the training methods. The data was
tested for normality and equal variances at the 5 percent confidence level (=0.05). Normality was tested using the
338
9
th

Anderson-Darling test. A resulting p-value less than 0.05 would indicate that the data exhibits a non-normal
distribution. A summary of the analyses is presented as Table 4.
Table 4: Statistical Summary of Percent Change in Production Rate

Based on the resulting p-values, the hypothesis of normally distributed data was rejected for zero-motion simulation
and on-machine training datasets. Therefore, Levenes test for equal variance was used since the data was not found
to be normally distributed. The resulting test statistic was 0.07 and corresponding p-value was 0.931. This p-value
is substantially greater than the 0.05 confidence limit, indicating that the hypothesis of equal variances be accepted.
The non-parametric Kruskal-Wallis test was employed to analyze the non-normall distributed data with equal
variances. The results of the single factor (training method) Kruskal-Wallis test are summarized in Table 5.
Table 5: Kruskal-Wallis Test of Percent Change in Production Rate

The Kruskal-Wallis test p-value was greater than the 0.05 confidence level, indicating that the hypothesis of a
common distribution be accepted. This leads to the conclusion that training method does not significantly affect
changes in production rate.
While training method did not significantly influence changes in production rate, it was also important to determine
whether each training method resulted in a significant increase in production rate. The non-parametric Sign Rank
test was selected as the appropriate analysis method based on the previous determination of non-normally distributed
data of equal variance. The results of tests for increased production rate are summarized in Table 6.
Table 6: Sign Rank Test Summary for Increased Production Rate

The p-values for the tests were less than the 0.05 confidence level, indicating that the hypothesis of no change in
production rate be rejected and the conclusion drawn that all training methods result in an increased production rate.
4.3 Confidence and Anxiety
Participants ranked their levels of confidence and anxiety immediately prior to performing the operational tests by
responding to the following statements:
339
9
th

1. I am confident in my ability to effectively load trucks with a wheel loader.
2. I feel anxious when I think about loading trucks with a wheel loader.
Responses regarding confidence and anxiety were collected on a five point Likert Scale, assigned a numerical value,
and are presented as Fig. 5 and Fig. 6, respectively. There was a general increase in confidence levels from pre- to
post-training. Following training there was no disagreement regarding confidence, and nearly all responses
indicated agreement to strong agreement with confidence in effectively operating a wheel loader. It is clear that
operator confidence increased during the course of the study, but data is not sufficient to conclude that the training is
the cause. Rather, it is possible, and perhaps even likely, that the increased confidence is a result of the previously
inexperienced operators becoming familiar and more comfortable with the loader during the course of the study.

FIG. 5: Operator Confidence Box Plot
It is difficult to discern a general trend in the anxiety data. Prior to training, the complete range of responses were
given and mean values ranged from indicating neutrality to disagreement. Following training, no participant
strongly agreed with feeling anxiety and mean values decreased only slightly. The greatest reduction in anxiety was
exhibited by those receiving on-machine training, with the average post-training response being disagreement.
Interestingly, responses from the zero-motion simulation group remained unchanged. While the confidence data
appear to indicate that the participants became familiar and comfortable with operating the wheel loader, it does not
indicate that anxiety levels were substantially decreased.
340
9
th

FIG. 6: Operator Anxiety Box Plot
5. CONCLUSIONS
The purposes of this study were to aid in understanding proper application of the ATS and provide preliminary data
regarding the impact of simulation-based training and the potential for producing improved operator performance
relative to traditional on-machine training. Based on the results obtained during this study, it was concluded that
full-motion simulation-based training using the Volvo ATS is a viable option for training wheel loader operators.
There was no statistically significant difference found between on-machine and simulation-based training. All
training methods resulted in statistically significant decreases in loading time and increases in production rate.
The results indicate that trainees responded better to full-motion simulation training than zero-motion simulation.
Training using full-motion simulation resulted in decreased levels of anxiety, while the anxiety levels of participants
receiving zero-motion simulation remained unchanged. Full-motion simulation also resulted in increased operator
confidence.
It is recommended that a hybrid training method be developed consisting of full-motion simulation-based training
augmented with on-machine training. The objectives of this hybrid training method should be clearly defined in
terms of desirable operational parameters and the training explicitly designed to achieve the specific objectives. The
learning objectives of the on-machine portion of training may include proper bucket loading techniques and
comprehensive operational safety.
The scope of this study was limited to participants who were not previously experienced in operation wheel loaders,
and the results obtained pertain only to operators at the very origin of the learning curve. It is hypothesized that
additional differences in the training methods may become evident if participants with operating experience are
studied.
6. REFERENCES
Anon. (2005). New Three-in-One Mining Equipment Simulator for Venetia. Mining and Quarry World, 2(2), 46.
Bullough, J. D., and Rea, M. S. (2001), Forward vehicular lighting and inclement weather conditions. Proc.,
Progress in Automobile Lighting Symp. (PAL 2001), Darmstadt University of Technology, Germany, 79-89.
Enriquez, M., Afonin, O., Yager, B., and Maclean, K. (2001). A pneumatic tactile alerting system for the driving
environment. Proc., 2001 Workshop on Perceptive User Interfaces, Lake Buena Vista, FL, 17.
Godley, S. T., Triggs, T. J., and Fildes, B. N. (2002). Driving simulator validation for speed research, Accident
Analysis and Prevention, 34(5), 589600.
341
9
th

Gokhale, S. (1995). Partnering with Industry to Develop a Training Simulator for Directional Drilling Equipment
Operators. Proc., 1995 ASEE Annual Conf., Anaheim, CA, 495-500.
Kamezaki, M., Iwata, H., and Sugano, S. (2008). Development of an Operational Skill-Training Simulator for
Double-Front Work Machine. Proc., 2008 IEEE/ASME International Conf. on Advanced Intelligent
Mechatronics, 170-175.
Mashiocchi, C., Dark, V., and Parkhurst, D. (2006). Evaluation of Virtual Reality Snowplow Simulator Training:
Final Report. Center for Transportation Research and Education (CTRE) Report 06-245, Iowa State Univ.,
Ames, IA.
Panerai, F., Droulez, J., Kelada, J., Kemeny, A., Balligand, E., and Favre, B. (2001). Speed and safety distance
control in truck driving: Comparison of simulation and real-world environment. Proc., Driving Simulation
Conf. 2001 (DSC2001).
Schneider, M., and Kiesler, S. (2005). Calling while driving: Effects of providing remote traffic context. Proc.,
Conf. on Human Factors in Computing Systems (CHI 2005): Technology, Safety, Community, Portland, OR,
561569.
Strayer, D., Drews, F., and Burns, S. (2004). The Development and Evaluation of a High-Fidelity Simulator
Training Program for Snowplow Operators. Utah Department of Transportation Report UT-04-17, Utah
Dept. of Trans., Salt Lake City, UT.
Tornros, J. (1998). Driving behavior in a real and a simulated road tunnel: A validation study. Accident Analysis
and Prevention, 30(4), 497503.
Wang, X., and Dunston, P. (2005). Heavy Equipment Operator Training via Virtual Modeling Techniques. Proc.,
2005 Construction Research Congress, San Diego, CA, 1241-1250.
342

BUILDING INFORMATION MODELLING

BIM Server: Features and Technical Requirements--------------------------------------345
Vishal Singh and Ning Gu

LEED Certification Review in a Virtual Environment-----------------------------------355
Shawn OKeeffe, Mohd Fairuz Shiratuddin and Desmond Fletcher

Changing Collaboration in Complex Building Projects
through the Use of BIM-------------------------------------------------------------------------363
Saskia Gabril

The Introduction of Building Information Modelling in Construction Projects:
An IT Innovation Perspective-----------------------------------------------------------------371
Arjen Adriaanse, Geert Dewulf and Hans Voordijk

Creation of a Building Information Modelling Course for
Commercial Construction at Purdue University------------------------------------------383
Shanna Schmelter and Clark Cory

9
th

BIM SERVER: FEATURES AND TECHNICAL REQUIREMENTS
Vishal Singh, PhD Candidate
The University of Sydney, Australia;
vsin9057@mail.usyd.edu.au
Ning Gu, Lecturer
The University of Newcastle, Australia;
Ning.Gu@newcastle.edu.au
ABSTRACT: Building Information Modelling (BIM) enables intelligent use and management of building information
embedded in object-oriented CAD models. The widespread use of object-oriented CAD tools such as ArchiCAD,
Revitt and Bentley in practice has generated greater interest in BIM. A number of BIM-compliant applications
ranging from analysis tools, product libraries, model checkers to facilities management applications have been
developed to enhance the BIM capabilities across disciplinary boundaries, generating research interests in BIM
servers as tools to integrate, share and manage the model developed in distributed collaborative teams. This paper
presents the findings from a project aimed at eliciting features and technical requirements for a BIM server. The
paper is based on: (a) a case study conducted with a state-of-art BIM server to identify its technical capabilities and
limitations, and; (b) analysis of technical features of current collaboration platforms (CPs) used in the architecture,
engineering and construction (AEC) industry. Analysis of data reveals that the development of BIM servers should
not be entirely for functional and operational purposes. Based on the findings the technical requirements are
classified into four main categories including both operational and support technical requirements (OTR and STR).
They are (a) BIM model management related requirements; (b) design review related requirements; (c) data
security related requirements; and (d) BIM server set-up implementation and usage assisting requirements.
KEYWORDS: BIM, BIM server, collaboration platform, operational technical requirements, support technical
requirement.
1. INTRODUCTION
Building Information Modelling (BIM) is an advanced approach to object-oriented CAD, which extends the
capability of traditional CAD approach by defining and applying intelligent relationships between building elements
in the model. BIM models include both geometric and non-geometric data, such as, object attributes and
specifications. The inbuilt intelligence allows automated extraction of 2D drawings, documentation and other
building information directly from the BIM model. This inbuilt intelligence also provides constraints that can reduce
modelling errors and prevent technical flaws in the design, based on the rules encoded in the software (Ibrahim et al.
2003, Lee et al. 2006). Most recent CAD packages have adopted the object-oriented approach with certain BIM
capabilities. A number of supporting applications have emerged that can exploit the information embedded in the
BIM model for model integration, design analysis, error checks, facilities management (FM), and so on (Khemlani
2007a). The emergence of multiple applications with the ability to directly use and exchange building information
between them provides opportunities for enhanced collaboration and distributed project development. BIM is
increasingly considered as an IT enabled approach that allows design integrity, virtual prototyping, simulations,
distributed access, retrieval and maintenance of the building data. Hence, the scope of BIM is expanding from the
current intra-disciplinary collaboration through specific BIM applications to multi-disciplinary collaboration through
a BIM server that provide a platform for direct integration, storage and exchange of data from multiple disciplines.
A BIM server is a collaboration platform (CP) that maintains a repository of the building data, and allows native
applications to import and export files from the database for viewing, checking, updating and modifying the data. In
general, a BIM server by itself has limited inbuilt applications. BIM servers are expected to allow exchange of
information between all applications involved in a building project life cycle including design tools, analysis tools,
FM tools, document management systems (DMS), and so on. In principle, BIM servers aim to provide collaboration
capabilities similar to DMS. However, while DMS are meant for collaboration through exchange of 2D drawings
and documents, BIM servers provide a platform for the integration and exchange of 3D model data with embedded
intelligence. This paper presents the findings from a project aimed at eliciting features and technical requirements
345
9
th

for a BIM server. Findings reported in this paper are based on: (a) a case study conducted with a state-of-art BIM
server to identify its technical capabilities and limitations, and; (b) an analysis of technical features of current CPs
used in the architecture, engineering and construction (AEC) industry. Analysis of data reveals that the development
of BIM servers should not be entirely for functional and operational purposes. Based on the findings the technical
requirements for a BIM server are classified into four main categories including both operational technical
requirements (OTR) and support technical requirements (STR). They are (a) BIM model management related
requirements; (b) design review related requirements; (c) data security related requirements; and (d) BIM server set-
up implementation and usage assisting requirements. Findings from this research will enhance BIM server research
and development to better facilitate the adoption of the technologies, leading to greater intelligent and automated
collaboration supports in design and construction.
2. BIM ADOPTION
Based on the literature (Johnson and Laepple 2003, Bernstein and Pittman 2004, Holzer 2007, Khemlani 2007b,
Howard and Bjork 2008) and focus group interviews (FGIs) with key industry players, we have identified the main
factors affecting BIM adoption (Gu et al. 2008, 2009). The perception and expectation of BIM against the industrys
current practice are summarized in terms of the following three main aspects: tools, processes and people.
1. Tools: Expectations of BIM vary across disciplines. For design disciplines, BIM is an extension to CAD. For
non-design disciplines such as contractors, BIM is an advanced DMS that can take-off data from CAD packages
directly. While there are evident overlaps, BIM application vendors seem to be aiming to integrate the two
requirements. The existing BIM applications are not yet mature for either purpose. Users with CAD
backgrounds, such as designers, expect BIM servers to support integrated visualization and navigation
comparable to the native applications they use. Users with DMS backgrounds, such as contractors, expect
visualization and navigation to be the features of BIM servers that are missing in existing DMS solutions.
2. Processes: BIM adoption would require a change in the existing work practice. An integrated model
development needs greater collaboration and communication across disciplines. A different approach to model
development is needed in a collaborative setting where multiple parties contribute to a centralized model.
Standard processes and agreed protocols are required to assign responsibilities and conduct design reviews and
validation. Experience from DBMS (Database Management System) will be useful for data organization and
management, but organizations will need to develop their own data management practices to suit their team
structure and project requirements. Different business models will be required to suit varied industry needs. A
BIM model can be maintained in-house or outsourced to service providers. In the latter case, additional legal
measures and agreements will be required to ensure data security and user confidence.
3. People: New roles and relationships within the project teams are emerging. Dedicated roles, such as BIM model
manager and BIM server manager will be inevitable for large scale projects. Team members need appropriate
training and information in order to be able to contribute and participate in the changing work environment.
In summary, as BIM matures it is likely to integrate the existing CAD packages and DMS into a single product. For
BIM to succeed and be adopted widely in the industry, all the stakeholders have to be informed about the potential
benefits to their disciplines and the project. Earlier research shows that (1) the lack of awareness, (2) the over-focus
on BIM as advancement of CAD packages only, and (3) the relative downplaying of BIMs document management
capabilities have inhibited the interest of non-design disciplines of the AEC/FM industry in BIM adoption. A user-
centric BIM research has to be more inclusive, since the success of BIM adoption lies in collective participation and
contribution from all the stakeholders in a building project. For this study, the above understanding of BIM adoption
shows that the development of BIM server technologies is not entirely for functional and operational purposes. BIM
servers should not only have the technological capability to support the collaboration requirements of diverse user
groups, but also provide adequate support features to assist the users in assessing, designing and implementing BIM.
3. RESEARCH METHOD
The lack of industry experience in the use of BIM servers means there is limited feedback from industry on technical
requirements for a BIM server (Gu et al. 2009). This study adopts a combined research method involving:
346
9
th

Case study using a state-of-art BIM server: A state-of-art BIM server is tested on a real-world project with
the following research objectives: (1) To test the current functionalities, usability and limitations of the
BIM server as a CP. (2) To use the result of (1) as a benchmark to propose features and technical
requirements for a BIM server that will address the adoption issues as discussed above.
Analysis of document-based collaboration platforms in the AEC industry: A review of current document-
based CPs and their use in the industry is conducted to identify features and technical requirements that
users may expect in a BIM server. This enables us to extract essential and relevant functions of
collaborative technologies that are currently in use, to be integrated into the ideal BIM server environment.
4. RESEARCH DATA AND ANALYSIS
4.1 Case study using a state-of-art BIM server
For the case study, a renovation project within an Australian landmark building is chosen. A BIM server:
EDMmodelServer and relevant BIM applications are used for space renovation and re-functioning of a service
chamber in the building. Therefore, the existing building data, such as the original design drawings, the existing
infrastructures in the service chamber, and its spatial relationships with other surrounding spaces become important
and increase the complexity of the project. These factors were considered and respected in the case study.
The two main tasks of this case study are (1) the construction of different discipline-specific models and (2) the
integration of the models as an integrated BIM model using EDMmodelServer. Three disciplines involved in the
case study are architecture, hydraulics, and lighting. Applications used for constructing these discipline-specific
models include ArchiCAD for the architectural model and DDS-CAD for the hydraulic and lighting models. Other
applications used within the case study include: Solibri Model Checker, Solibri Model Viewer, DDS-CAD Viewer
and Octaga Modeller (a plug-in for EDMmodelServer for 3D model viewing). The case study tests a wide variety
of issues including building data visualization, analysis and collaboration. Specific features that are tested include
(1) object attributes in the discipline-specific models; (2) intelligent relationships within a discipline-specific model
and between different models in the integrated BIM model; (3) data representation, visualization and access
functions; (4) analysis functions that focus on design analysis and model evaluation; (5) project collaboration and
communication functions.
In general, the BIM server has been found useful for design collaboration. The BIM server has well developed
features for data upload, model integration and information extraction such as report generation. However, some
technological and implementation issues are identified during the case study, which are likely to scale up and
develop into a major roadblock in adopting the server for larger projects. These issues are primarily related to:
Set-up and access to the BIM server: The model server allows defining access rights and permissions based
on participant roles and responsibilities. In the case study few participants were involved. Even in the simplified
scenario, interventions were required midway through the project and personal meetings were organized to
coordinate activities. In a full-scale project, roles and responsibilities not only are likely to increase in
complexity, but also overlap. Thus, it will be useful to methodologically identify role dependencies and
responsibilities, which are critical to the set-up and access to the integrated BIM model.
Help function and tutorials: Although the participants in the case studies had some training and familiarity
with the BIM server, during the case study some difficulties were reported in tool usage and technical support.
Participants emphasize the need for improved help functions and tutorials. There are suggestions for open-
source training materials where users can learn form each others experience.
User interface (UI): The UI of the BIM server is reported to be confusing. Users that have extended experience
with native disciplinary applications (e.g. CAD tools), find the BIM server interface non-intuitive. Users from
different disciplinary backgrounds may have different usage patterns, requiring different standard interfaces.
Currently, different data and information are shown and limited in one single window. A flexible, user-friendly
UI that allows customization to suit different user profiles is required. 3D visualization of the plug-ins for
viewing the BIM model is good. However, further tests are required for project with larger data sets.
Data management and modification: In the BIM server, objects and its operations are well defined. However,
during model integration, object duplication or model conflict may occur especially when the same object is
347
9
th

created in parallel in different discipline-specific models. Data back-up is critical, especially in the likelihood of
the crash of the BIM server. The upgrades of the BIM server are organized in downloadable modules. If a new
version is available, a message from the application provider will notify the users when they login to the server.
Extended functionalities for project communication: Some other features and technical requirements
reported as wish-lists in an earlier study (Gu et al. 2009) were reiterate, with most suggestions related to
improving project communication. For example, although the BIM server allows object tagging and change
notifications, if an object is tagged on the BIM server, and the tagged model is downloaded, the tagged message
is not downloaded. The user needs to manually check and update the tagged object separately in the native
disciplinary model. Flexibility to download the appended project communication records along with the model
may be useful. It was also suggested that the BIM server should provide more synchronous communication
features such as an embedded instant messenger for the ease and clarity of project communication.
4.2 Analysis of document-based collaboration platforms in the AEC industry
The analysis includes the examination of existing online CPs, primarily DMS, such as Incite, Aconex, TeamBinder
and Project Centre, along with interviews with industry partners who have adopted the technologies in their
practices. The review was conducted to understand the implementation and application of web-based CPs in the
AEC industry. Such collaborative practices exist within the industry and therefore, they may act as a gauge for using
BIM servers as CPs. The analyses suggests that
The initial CP set-up is complex. Various dependencies within the process and between activities and people
need to be identified before its operation. The complexity in setting up a BIM server as a CP for building
project development may be even greater because model-based data exchange will require greater coordination
owing to larger size of the data set, varied file formats and tool compatibility issues.
There are various levels of DMS usage. In some projects, DMS is used across the entire project lifecycle,
involving most of the project participants. In other cases, only some of the project participants coordinate their
activities through DMS, or the DMS is used in specific stages only. This scoping of DMS as a CP is usually
conducted at the initial phases of the project. Such scoping is also critical for BIM server adoption.
In the current practice, a customized project instruction document is generally developed to serve as a guide for
the project operation. This ensures that as the project develops and the team dynamics change, key
terminologies and standard procedures are agreed and complied by all participants.
In general, DMS automate the process of uploading, validating, approving and distributing documents. A series
of business rules encoded within the applications at the start of the project automates the decision making such
as which folder to upload documents to, how to validate the document, who to distribute documents to (based
on a distribution matrix), and so on. The inbuilt intelligence in form of business rules and distribution matrix
would require knowledge elicitation from project partners.
Once the protocols are established, the DMS is configured for the project. The administration team and project
partners are trained, before the system goes online.
Training programs are available for all functional levels, from the standard tools that all project teams will use,
to the more advanced construction project management tools such as workflows and tender modules. Help is
provided in form of manuals, technical support and video demonstrations.
The project administrator requires in-depth knowledge of the required document flow process as well as basic
configurations and user requirements on the DMS. In general, a company administrator is appointed to
coordinate with the project administrator and has access rights similar to project administrator.
Communication is a critical part of all DMS. Most DMS provide multiple modes of communication including
instant messages, SMS texts, emails and voice communications. They support both synchronous and
asynchronous communications, and are used for multiple purposes, such as direct project communication,
documentation, as well as sending notifications, reminders and clarifications.
348
9
th

5. TECHNICAL REQUIREMENTS FOR A BIM SERVER
Based on the case study and the analysis above, the authors have broadly grouped the features and technical
requirements for BIM servers as operational technical requirements (OTR) and support technical requirements
(STR). Depending on the actual project, there may be overlaps between OTR and STR.
OTR refer to the features and technical requirements needed during the usage of the BIM server in direct
support for a building project. OTR can be further divided into the following three categories.
BIM model management related requirements: These are directly related to the storage, operation and
maintenance of the BIM model that contains all the building information.
Design review related requirements: These are specifically related to design review activities, including
various functions needed for design visualization and navigation, as well as team communication.
Data security related requirements: These features and technical requirements are related to network
security and the prevention of unauthorized access into the system.
STR, such as, help menus and FAQs, have been recognized as an integral part of technological tools (Dicks and
Lind 1995) and are likely to be critical to technology adoption. In project collaboration tools, such as some of
the DMS (e.g. Aconex, Team Binder, Project Centre, and Incite), support features to facilitate the set-up and
implementation of the CPs include assessment matrices, templates, etc. Hence, besides the common support
features such as help menus, tutorials, and FAQs for facilitating the usage of the BIM server, the other
important part of STR for a BIM server includes project decision support features that facilitate and assist the
set-up and implementation of the BIM server for a particular building project.
BIM server set-up, implementation and usage assisting requirements: These refer to the functions that
are expected to facilitate and assist the set-up, implementation and usage of the BIM server.
Table 1 illustrates the above classifications with some possible overlaps in between. Sections 5.1 to 5.4 elaborate
each of the features and technical requirements that are listed in Table 1.
Table 1 Features and Technical Requirements for a BIM Server
Features and Technical Requirements
(a) BIM model management related requirements
BIM model organization
Model repository
Sub-models, and objects with different levels of details
Public and private model spaces
Globally Unique Identifier (GUID) for all object data
Information Delivery Manuals (IDM) based specifications
Model access and usability
Secured log-in with access rights
Hierarchical model administration structure
Download/upload model, and check- in/ check-out/ check-out with lock
Version lock and archiving
Model viewing options
Documentation and reports
User Interface (UI)
Customizable interface
Online real-time viewing, printing and markups
On-click object property check and modification
(b) Design review related requirements
Design visualization and navigation
Team communication and interaction
(c) Data security related requirements
349
9
th

Features supporting confidentiality, integrity, and availability
System security
User authentication
Data security
Access control
Encryption
(d) BIM server set-up, implementation and usage assisting requirements
Project decision support
Project scoping support
Software tool compatibility matrix
BIM scoping support
Server administration support
System configuration manager
System configuration layout viewer
System status viewer
Help support and training
Legal and contractual support
5.1 BIM model management related requirements:
5.1.1 BIM model organization
Features and technical requirements for model management and organization should include the following.
Model repository: A BIM server should provide a centralized data repository for the building project. This data
repository can be linked to other federated data repositories to increase data capacity and efficiency.
Hierarchical model structure: A BIM model on a server is organized in a hierarchical structure. For example, at
present the model-tree in EDM has the following hierarchy: project > site > building > building storey.
However, users may want a different model structure based on their requirements, e.g. a client may want to
group projects within a site rather than the other way round, i.e. site > project > building > building storey, and
so on. Thus, BIM servers should support the flexibility to customize the model structure.
Sub-models, and objects with different levels of details: the BIM server should provide the ability to map
objects with different levels of details. For example, if level 1 detail only shows a rectangular volume for a
room, the level 2 detail of same volume may show all the openings, and doors and windows. Users should be
able to navigate and switch across the different views through simple functions and shortcut keys. In order to
support such functionalities, mapping of objects with different levels of details will be required.
BIM server should support the ability to store and present objects of the model as text-based information in
repositories, and link 3D object-model with model viewer. Choosing object in one window (text-based model
tree or 3D model in model viewer) should highlight the corresponding data in the other window.
Object and model history, such as ownership and modification records, should be maintained in the repository.
Object property: The BIM server should provide the ability to overlay additional object properties. For
example, Quality of survey may not be a default object property. If this is the case, in the BIM server, this can
be an overlayed property linked with each object. Technical issues may arise if the data is downloaded and
uploaded again. Additional technical measures will be required to deal with such issues.
Public and private model spaces: BIM server should allow differentiation of public and private model. Public
model is accessible to all users with access rights. Private model could be model in progress, but not ready to be
shared with others. However, private models may be shared with a select group of users.
GUID and IDM: Globally Unique Identifier (GUID) allows each object to be uniquely identified, preventing
duplication. Information Delivery Manuals (IDM) details specifications and approaches for connecting the BIM
350
9
th

approach with business processes. BIM servers should enable efficient integration of GUID and IDM to deal
with problems encountered in merging different discipline-specific models.
5.1.2 Model access and usability features
Some of the features and technical requirements related to model usability are discussed below.
Secured log-in with access privileges: it should be possible to import roles and personnel data from
information flow dependency matrix generated in the BIM server set-up phase. See section 5.4 for details.
Hierarchical model administration structure: BIM server administration deals with management and
allocation of model access rights, data control and security. Typically, hierarchical administrative structures
exist across distributed teams and in large organizations to manage day-to-day local and global issues. Thus,
BIM sever should allow administrative structures that reflect and support existing organizational practices.
Download / Upload model: Various modes of interaction for model download/upload are possible to include
download/upload buttons as well as drag and drop options. Ability to download/upload data straight to/from an
email account, which is possible in existing DMS when dealing with documents, will be useful.
Check-in / check-out and version lock: Check-in options should allow addition of new partial model or
merging with existing model. Again, different modes of interaction are possible to include buttons and drag and
drop capabilities. Similarly, check-out options should allow download of complete model or partial model using
different modes of interaction. A check-out with lock feature should be provided to notify other users that the
checked-out data has been locked and deemed not usable. A version lock feature should be provided to lock
version of the model after sign-off, as a form of archiving.
Model viewing options: BIM server should support the ability to capture and save screen shots, which is a
standard functionality provided across CAD packages. Other features, such as the option to choose the level of
detail for viewing should be available, i.e. sub-sets should be managed such that users can choose the level of
detail for viewing by selecting options on a checklist, e.g. conceptual block model, space layout model, etc.
Documentation and reports: When downloading a part model from BIM server, there should be options to
generate reports on parametric, linked, and external information for selected objects and the other objects in the
rest of the model. This information can be in form of a checklist, where users can choose to get details of only
those objects they intend to modify, delete or replace. Ideally, a facility to append this information to objects
will be helpful, but that would be useless until the native applications can receive those additional data.
BIM servers should support the ability to generate and export PDF or other document formats. This capability
also allows direct offloading of ready to use information to DMS, in which case some users may not need to
access the BIM server. They can continue interacting with DMS as they have been doing at present.
BIM servers should support the ability to integrate information from product libraries. It should be possible to
create a comparison report for alternative product options.
Features should be provided to validate rules while uploading the files on the BIM server. Users should have
the option to switch validation check on or off.
Technical provisions are required for data ownership transfer and handover in a BIM server. These
functionalities should account for security measures to deal with such change of hands, log-ins and passwords.
5.1.3 User Interface (UI)
Other than the standard UI features, the BIM server interface should include: (1) Model tree view and 3D viewer
position; (2) support for online real-time viewing, printing and mark-ups; (3) the ability to click on an object, and
check what sub-sets it belongs to, and (4) the ability to click on an object, and switch between the sub-sets it belongs
to, for another sub-set selection. Users should also be able to customize and choose the available UI functionalities.
5.2 Design review related requirements:
The following basic features and requirements are related to distributed design review.
351
9
th

Team communication and interaction: Distributed design reviews may require parallel video conferencing
and similar interaction media. BIM servers should be compatible with such technologies, and integration of
BIM servers with these technologies will be useful for model navigation and viewing. Organizations may want
to maintain a record of the design review interactions in the repository. Thus, BIM servers should provide the
ability to capture real time interaction data from meetings and online reviews. Some of these interaction
platforms, such as instant document/ message exchange window, may be directly hosted on the BIM server.
Design visualization and navigation: Building projects often result in large data files, which reduce the online
navigation and viewing capabilities. Hence, for effective design review across distributed teams, the 3D Model
viewers supported by BIM servers should have high data compression capabilities while maintaining the image
quality. It will be useful to provide technical features that allow instant, online mark-up, and tagging on a shared
document, model or object being viewed by design reviewers and users.
5.3 Data security related requirements:
Security of data on BIM server should account for confidentiality, integrity and availability of data. These features
and related technical requirements are discussed below.
Confidentiality: The data stored on the BIM server should be available to authorised users only, and on the
need-to-know basis. This service is crucial to secure sensitive data from malicious intruders.
Integrity: All BIM data should be only created, modified and deleted by authorised users having an authorised
access, and should be a subject to integrity cross checking.
Availability: Data and services provided by a BIM server should be available to users when they need them. As
BIM servers have a role of a central data repository serving simultaneously a range of users involved in a
project, availability requirements are of greater consequence than in specialised systems.
System security: BIM servers should have user authentication facilities to ensure that only authorised users
can access it. The current BIM servers only partially satisfy the security requirements. While they would
typically provide for internal user authentication in form of log-in and password, they fail to provide other
forms of authentication. Moreover, if sufficient password management is not put in place, by the system and
users alike, such authentication procedures may be regarded as inadequate.
Data security: BIM server should provide effective data access control, with access privileges to individual
pieces of data, including create, delete, read, write and execute. While access control is typically implemented
by the tested BIM server, it does not provide for fine granularity. It would be desirable for BIM servers to adopt
the Role Based Access Control (RBAC) (Sandhu et al. 1996), which is already used by leading Database
Management Systems (DBMS). RBAC allocates access privileges to roles, rather than users, which greatly
simplifies the privilege management task, especially in such a dynamic environment as BIM. The BIM data
should be protected by means of encryption, both when stored on the system and transferred over a network.
This feature is available in major DBMS, it should be adopted by BIM servers.
5.4 BIM set-up, implementation and usage assisting requirements:
5.4.1 Project decision support
The project decision support consists of three main requirements: (1) Project decision support to identify project
dependencies in terms of people, processes and resources, (2) Software tool compatibility matrix for selection of
tools to be used by each collaborating partner contingent on project requirements and technological capabilities and
limitations of dependent collaborators, and (3) BIM scoping support to decide on BIM approaches contingent on
project requirements and technological capabilities of collaborators.
Project scoping support: Initially workflow process maps should be developed. Various forms would be a
useful way to gather these data from project team members and the client. These should be accessible online
through a web-based interface with secured log-in. Users should be able to download the forms, work offline,
and upload them when they are filled. Users should be able to save partially filled forms and complete and
submit the same in multiple sessions. Once the data is collected, ideally, users should be able to generate
dependency matrices (Yassine and Braha 2003) automatically, with flexibility to develop/ modify these
352
9
th

manually. Graphical representation of the dependencies for easy comprehension and viewing, and text-based
search of required dependency is desirable. For example, if the user enters an activity such as design review, the
viewer should show various dependent activities and people, based on what dependency is being sought to
observe. Dependencies can be based on activities, people or resources. Once the dependencies are approved,
there should be a feature to allow setting up automated reminders and notifications. These notifications should
be sent across the medium of communication preferred by the target user, i.e., fax, email, etc.
Software tool compatibility matrix is required on the BIM server, which is accessible online, and provides
details of tools in terms of compatibility, data formats and interoperability, and capabilities. A directory of tool
experts, and related online tutorials and FAQs can be linked to the matrix. Project partners can coordinate tool
selection by applying the matrix.
BIM scoping support: In a typical project there are various levels of roles and associations. Some of the
personnel in the project may not have direct access to the BIM server for various reasons. Also, in order to
successfully use the BIM server as a CP it should be possible to receive and upload building information to the
BIM server through media such as email and fax that are preferred by different users. Such technical capability
is supported by document-based CPs, and, hence, similar expectations exist for the BIM server as a CP. These
requirements enhance the scope of BIM usage in a project. Some technical capabilities such as ability to capture
real-time data from site are also important for on-site/ off-site project coordination.
5.4.2 Server administration support
A BIM server integrates with other tools such as CAD, analysis tools, other discipline specific applications, DMS,
etc. This integrated system should be flexible enough to configure differently to suit different project requirements.
System configuration manager should ensure that the system configuration complies with project
dependencies, allows interactivity between models, documents and appended information. The system
configuration manager should have a customizable UI. It should support rules that regulate the information
provided to be suitable for, and easily adopted to needs of: (1) different users including designer, contractor,
facility manager, client and so on, (2) different building project life cycle ranging from project identification to
bid, start up, design, contract and operate, and (3) different scales of collaborative projects. Synchronous
communication, e.g., chat room, videoconference, and asynchronous communication, e.g., broadcasting and
email, should be supported to improve the project communication.
System configuration layout viewer is required that graphically shows how different types of data are linked.
Similarly, system status viewer is required for notification of errors, activities update, update on system
performance, and user status, e.g., how many users are logged-in at a given time.
Data change register is required to maintain the history of the changes made to the data.
The administration support should also facilitate report generation, data back-ups and archives, either manually or
through a pre-set default value (time or size) for automated activation.
5.4.3 Help support and training
Help support and training is critical to the use of the BIM server as a CP. Training support varies with the roles and
responsibilities of the users. Various types of training materials and approaches can be used in conjunction such as:
(1) traditional training and support tools that include help menus, FAQs and helpdesk. (2) open-source training
materials such as technical support blogs that maintain threads of earlier complaints and resolution methods reported
by users and experts. (3) Project-wiki (Kalny 2007) can be created on the server to share project information and
tool usage information. (4) Interactive tutorials such as those already available from various proprietary tools. (5)
An expert directory maintained in the tool-compatibility matrix can provide points of contact for training support.
5.4.4 Legal and contractual support
The model development, reviewing, uploading, downloading, and analysis activities could be quite complex within
an integrated BIM server environment. Specifying ownership, updating liabilities and responsibilities would need
careful consideration. A BIM server use contract agreement is required which should be signed and agreed upon by
the project partners at project initiation. Business rules are needed as a technical feature for model management and
data organization such as archiving, record keeping, backups, and so on. It should be possible to automatically check
353
9
th

if these rules conform to IDM specifications, a comprehensive document that details the approach to using and
developing a BIM model. Among other aspects, the legal and contractual support should ensure that the agreements
account for: (1) intellectual property agreements and policies for data exchange, (2) classification of public and
private data, and (3) correspondence protocols.
6. CONCLUSION
This paper presents features and technical requirements for a BIM server. The study shows that the development of
BIM server technologies should not be entirely for functional and operational purposes. BIM servers should not only
have the technological capability to support the collaboration requirements of diverse user groups, but also provide
adequate supporting features to assist the users in assessing, designing and implementing the BIM approach. As a
result, the developed features and technical requirements have been broadly grouped as OTR and STR. Three
categories of OTR: (a) BIM model management related requirements, (b) design review related requirements, (c)
data security related requirements, and one category of STR: (d) BIM server set-up implementation and usage
assisting requirements are subsequently presented and discussed. Findings from this research will enhance BIM
server research and development to better facilitate the adoption of the technologies, leading to greater intelligent
and automated collaboration supports in design and construction.
7. REFERENCES
Bernstein, P.G. anad Pittman, J.H. (2004). Barriers to the adoption of Building Information Modeling in the
building industry, Autodesk Building Solutions.
Dicks, R. S., and S. Lind. (1995). Logistics of integrating online help, documentation, and training: a practical
example. Proc. of 13th ann. int. conf. on Systems documentation, Savannah, Georgia, US: 34-38.
Gu, N., Singh, V., Taylor, C., London, K. and Brankovic, L. (2008). BIM: Expectations and a reality check,
Proceedings of ICCCBEX11 & INCITE 2008, Tsinghua University Press, China.
Gu, N., Singh, V., Taylor, C., London, K. and Brankovic, L. (2009) BIM adoption: expectations across
disciplines. In: Underwood, J and Isikdag (eds), J Handbook of Research on Building Information
Modelling and Construction Informatics: Concepts and Technologies, IGI Publishing (to appear).
Holzer, D. (2007). Are You Talking To Me? Why BIM Alone Is Not The Answer. In: Proc. of 4th ICAASA.
Howard, R. and Bjork, B. (2008). Building information modelling: Experts views on standardisation and industry
deployment. Advanced Engineering Informatics 22: 271280.
Ibrahim, M., Krawczyk, R. and Schipporiet, G. (2003). CAD smart objects: potentials and limitations, ECAADe
21: Digital design: 547-552.
Johnson, R.E. and Laepple, E.S. (2003). Digital Innovation and Organizational Change in Design Practice. CRS
Center Working Paper no. 2, CRS Center, Texas A&M University.
Kalny, O. (2007). Enterprise Wiki: An Emerging Technology to be Considered by the AEC Industry. AECbytes
Viewpoint 31, March 19, 2007.
Khemlani, L. (2007a). Supporting Technologies for BIM Exhibited at AIA 2007. AECbytes, May 24, 2007.
Khemlani, L. (2007b). Top Criteria for BIM Solutions: AECbytes survey results. AECbytes October 10, 2007.
Lee, G., Sacks, R. and Eastman, C.M. (2006). Specifying parametric building object behavior (BOB) for a Building
Information Modeling system. Automation in Construction 15: 758 776.
Sandhu, R.S., Coyne, E.J., Feinstein, H.L. and Youman, C.E. (1996). Role-based access control models.
Computer 29 (2):38-47.
Yassine, A. and Braha, D. (2003). Complex Concurrent Engineering and the Design Structure Matrix Method.
Concurrent Engineering 11:165-176.
354
9
th

LEED CERTIFICATION REVIEW IN A VIRTUAL ENVIRONMENT

Shawn E. OKeeffe
shawnokeeffe@mac.com

Mohd Fairuz Shiratuddin, Ph.D.
mohd.shiratuddin@usm.edu

Desmond Fletcher
desmond.fletcher@usm.edu

ABSTRACT: The excessive use of energy, water, and regional materials are becoming a worldwide concern. This
has led to councils around the world to develop standards and guidelines for review processes to help manage these
global concerns and promote the design and development of sustainable built environment. These guidelines and
standard review processes are used to reduce the depletion of natural resources, and to make the world healthier
and more sustainable for future occupants. The authors propose a new and innovative way of managing and
harvesting sustainable data for the purpose of LEED (Leadership in Energy and Environmental Design)
certification review. Conventionally, changes to the LEED certification review are sometimes made to compensate
for over expenditure of revenue, but these changes could not be analyzed early in the project life cycle. The authors
is developing a 5D BIM model that allows the viewing and reviewing of LEED information in a VE (Virtual
Environment), allows the analysis of cost to be interoperable with the BIM and the LEED certification review
process, and also allows the analysis of how changes made over time affect the total cost of the LEED review.

KEYWORDS: 5D, BIM, LEED, virtual environment.

1. INTRODUCTION

Around the world, people are concern about energy, water, carbon footprint, indoor and outdoor air quality,
harvesting of regional material, and disposal of waste. Buildings around the world affect the livelihood of all living
species through their consumption of energy, resulting to pollution and ozone depletion. Buildings are also
consuming 5 billion gallons of potable water per day (Krygiel et al, 2008). In the USA, buildings consume
approximately 37% of the world energy, and 68% of the world electricity (LEED-NC V2.2, 2007). For LEED
certified green buildings, Capital E (a premier provider of strategic consulting, technology assessment and
deployment, and advisory services to firms and investors in the clean energy industry), has reported an average
energy savings of 30%, average carbon reduction of 35%, a savings of 30-50% on potable water use, a reduction of
land filled waste of 50-97%. In 2003, Capital E developed an average first cost premium of 2% based off 33 LEED
certified buildings in California (Krygiel et al, 2008).

Selecting regional materials and developing buildings that are sustainable is a huge change for the AEC industry.
Sustainable buildings are not something new, but only recently have governments made sustainable development
mandatory. In 1987, the World Commission on Environment and Development stated to the United Nations that
sustainable development must meet the needs of the present without compromising the ability of future generations
to meet their own needs. A good analogy and example of sustainable design is the Native American teepee (Krygiel
et al, 2008). The teepee is designed using regional materials and it does not compromise the future of the land,
deplete the natural material sources, and the people who will inhabit that land. Teepee materials are recycled back
into the native environment without waste. Currently, a typical USA construction project generates 2.5 pounds
(~1.13 kilograms) of solid waste per square foot of floor area. Forty percent of total waste in the USA is caused by
355
9
th

construction and demolition, and LEED is helping some project to achieve an 80% waste diversion rate (LEED-NC
V2.2, 2007).

The statistics bring forth a need to provide an efficient way to review sustainability information using Building
Information Modeling (BIM). In this paper, the authors propose the use of Virtual Environment (VE) and 5D-BIM
concepts for LEED certification review. The authors envisioned that this approach would reduce the duration of
certification process, provide certification information that can be achieved in a timely manner, and able to select
different types of material and how they affect cost. By doing so, the AEC industry in general and project
stakeholders in particular, can benefit from better tracking and forecasting of sustainability information, and
construction events early on in projects life cycle. By reviewing specific LEED credits within a VE, a standard
review process working in parallel with the BIM model can be achieved, unlike the traditional paper-based LEED
review method for certification.

2. 5D-BIM PROTOTYPE FOR LEED CERTIFICATION REVIEW IN VE

The 5D-BIM is a constructiontime-simulation model of a virtual building that contains cost and other project
related information (Jernigan, 2008). 5D is the linking of the design and construction model (3D), schedule (4D),
and cost (5D). When utilizing a 5D model, users can change any aspect of the three-dimension (3D), with the other
two remaining dimensions (4D and 5D) be automatically updated to accommodate the change (Jernigan, 2008). 4D
and 5D allow AEC professionals and the project owner to utilize simulations to ensure a project plan is feasible and
efficient (Eastman et. al., 2008). The benefits of using a 5D model include: the improvement in communication and
collaboration among stake holders; and better analysis of site logistics, trade coordination on site, and comparisons
of schedule and tracking of construction progress (Eastman et. al., 2008).

A prototype 5D-BIM model is being developed using a suite of software by Vico Software. Sustainability
information for LEED certification purposes is furnished and included in the prototype 5D-BIM model. Besides
being utilized for LEED certification, the sustainability information can be used for GREEN building record such as
construction material usage and cost, duration of which GREEN materials are installed, CSI (Construction
Specifications Institute) Uniformat Methods of Installation, Task Progress of Material Installation, and when the
materials are completely installed (see figure 1). The 5D-BIM model can also be used for Earned Value Analysis
(EVA). The EVA allows the user to simulate what-if scenarios and see how cost is affected over time. Inside the
EVA viewer, the 5D cost related information includes Labor, Material, Sub-contract, Equipment and Other, (see
figure 2). Reports of all EVA query can be exported to Microsoft Excel spreadsheet that allows for further
interoperability and data sharing with other software tools. Users can navigate in real-time through the 5D-BIM
model and access the LEED sustainability information.

For LEED certification review performed within a VE to be successful, a LEED-AP
1
or designated person by the
Owner must have an early relationship with the AEC professionals involved in the project, specifically the designers
during the early phases of the project life cycle. Currently, to certify a project, a LEED-AP does not have to be hired
or recognized as the party to make submittals to the Green Building Certification Institute (GBCI). A non-accredited
project member can submit to the GBCI if appointed by the Owner to do so. A LEED-AP certifying a project should
have experience and have officially certified a Green Building meeting the standards given and developed by the
USGBC.

1
A LEED-AP is a person who has passed the LEED-AP exam given by the Green Building Certification Institute (GBCI), a third party
organization that handles the accreditation process for the USGBC. A LEED-AP is accredited by the USGBC to streamline and complete the
LEED certification process.
356
9
th

FIG. 1: Prototype of the 5D-BIM model

FIG 2: EVA of the prototype 5D-BIM model

A LEED Project can be certified as one of four different levels of certification depending on how many points are
achieved from the GBCI. The four levels of certification are: Certified; 26-32 points, Silver; 33-38 points, Gold; 39-
51 points, and Platinum; 52-69 points. Type of LEED Project certification depends on what category the project falls
under: LEED for New Construction (NC) and Major Renovation, LEED for Neighborhood Development (ND),
LEED for Schools, LEED for Existing Buildings, LEED Core and Shell, and LEED for Existing Buildings
Operation and Maintenance. The prototype for this paper utilizes the LEED NC Version 2.2. There are over 900
certified buildings and almost 7,000 more registered seeking certification as of June 2007 (LEED-NC V2.2, 2007).

Figure 3 shows a workflow diagram for a successful inclusion and use of LEED information in a 5D-BIM model.
The LEED information can be viewed and reviewed in real-time within a VE. To include the LEED information into
the 5D-BIM model, a new custom recipe must be created for the model's elements (see figure 4). The recipe is then
saved in the standard and project database and will now contain not only the LEED information, but also labor
LEED
Information
Uniformat
Cost
Cost changing over time
357
9
th

methods, cost, and sustainable resources. The standard and project databases allow the model element's recipe to be
interoperable among the Vico software suite. The authors used the Vico software suite to produce an interoperable
information workflow for a LEED-ready 5D-BIM model; i.e. between the 3D model, the estimate, the schedule and
the cost analysis (see figure 5). Figure 5 shows the relationship between cost and LEED credits viewable via the
Vico Cost Explorer Software. The LEED-ready 5D-BIM model can then be viewed and reviewed within a VE using
the 5D Presenter software. In the 5D Presenter, users can access and visualize the cost, the LEED information for
certification, the 4D schedule, and the Earned Value Analysis (EVA). Users can also watch the 5D-BIM model
develop over time (see figures 1 & 2).

1) In Constructor recipes are assigned to model elements and the model must be linked to the project database and
standard database.
2) An Estimator project is created and linked to the project database and standard database.
3) The link is verified to ensure connection between Constructor and Estimator.

4) In Estimator, the model data is exported to Cost Explorer.
5) The link is verified to ensure connection between Estimator and Cost Explorer.

6) In Constructor, the WBS (Work Breakdown Structure) is used to create project Task and Task Summary.
7) In Control, a new project is created.
8) In Constructor, the model data is published to Control.

9) In Constructor, the LEED-ready 5D-BIM model is published to a VE in 5D Presenter.

FIG 3: The workflow

The information necessary for the review of LEED certification is provided by the USGBC. The authors have
embedded LEED information into the model element parameters so that the model element parameters contain a
credit number that can be referenced to a digital or paper-based LEED summary checklist. The checklist contains
the credit title, intent of credit, requirements, strategy, and submittal documents. Each credit number has a specific
number of points. Using the LEED-ready 5D-BIM model for certification, users are allowed to access the LEED
credits and points in real-time in the VE. The LEED certification process can be done in a VE if the Credit Numbers
and requirements are embedded into the BIM model during the Design Development Phase of the Project Life
Cycle.

Constructor Estimator
1
Estimator Cost Explorer
4
2
3
5
III
.
II
I
Constructor Control
6
7
5
IV
Constructor 5D Presenter
9
8
LEED Recipes
created
358
9
th

FIG 4: Development of recipes

FIG 5: LEED cost analysis

The authors believe that it is necessary to innovate and improve the LEED certification process. LEED review
within a VE allows the AEC industry to move forward to a more integrated approach. The review for LEED
certification in a VE will help the AEC manage sustainability information when utilizing a BIM model, and the BIM
model can supplement the certification process. LEED certification in a 5D-BIM model provides various benefits
including the reduction of cost of sustainable design, providing various "what-ifs" scenarios to achieve better
sustainable designs, improving real-time communication of certification intents, and reduction of the time wastage
that occurs when waiting for submittals to be approved for certification. Overall, this could lead to healthier and
more efficient built environments. The authors are still working and improving the LEED certification process in a
VE that will lead to a more automated approach. Current ongoing work includes proposing a sixth dimension (6D)
which is energy (see figure 6).

LEED Credit and Points
LEED Recipe
and Uniformat
cost analysis
bubble
diagram.
359
9
th

FIG 6: Samples of energy analyses based on the current research work

The idea of new dimensions for the AEC industry is not new. Lee, et al. (2005) proposed nD modeling as an enabler
for construction improvement. An nD model is an extension of the BIM that incorporates multi-aspects of design
information required at each stage of the overall project life cycle. The authors are proposing a new 6th dimension
for the AEC industry. The proposed new dimension is energy and its purpose is to assess the energy cost, energy
standards, and energy efficient quality of all AEC related projects to provide humans with healthier and more
sustainable built environments. Figure 7 below partially shows how 6D can be directly related to 2D, 3D, 4D, and
5D, and also shows the direct link between LEED certification and 6D. The authors believe 6D is a vital and
sustainable part of nD modeling, sustainable design and construction, and BIM.

FIG 7: Proposed 6D
4. REFERENCES

Eastman, C., Teicholz, P., Sacks, R. and Liston, K. (2008), BIM Handbook: A Guide to Building Information
Modeling for Owners, Managers, Designers, Engineers and Contractors, Publisher: Wiley, ISBN-10:
0470185287, ISBN-13: 978-0470185285.

Jernigan, F. (2008), BIG BIM little bim - the practical approach to building information modeling - Integrated
practice done the right way!, Publisher: 4Site Press; 2
nd
edition, ISBN-10: 0979569923, ISBN-13: 978-
0979569920.

360
9
th

Krygiel, E., Nies, B. and McDowell, S. (2008), Green BIM: Successful Sustainable Design with BIM. Publisher:
Sybex, ISBN-10: 0470239603, ISBN-13: 978-0470239605.

Lee, A and Aouad, G. and Cooper, R and Fu, C and Marshall-Ponting, AJ and Tah, JHM and Wu, S. (2005), 'nD
modelling - a driver or enabler for construction improvement?', RICS Research Paper Series, RICS, London,
5 (6) , pp. 1-16.

LEED-NC V2.2 (2007), USGBC: LEED for New Construction. Retrieved on July 15, 2009 from
http://www.usgbc.org/DisplayPage.aspx?CMSPageID=220.

361
9
th

CHANGING COLLABORATION IN COMPLEX BUILDING PROJECTS
THROUGH THE USE OF BIM
Saskia Gabril, PhD,
Henry van de Velde Institute, Artesis University College of Antwerp, Antwerp, Belgium;
Saskia.Gabriel@artesis.be; http://www.artesis.be
ABSTRACT: Generally, the term Building Information Modeling (BIM) is used in many ways. This paper intends to
give an overview of the current meaning of BIM, as well as to identify the currently experienced advantages and
disadvantages of BIM in the building industry. First, a literature study was performed to investigate which
advantages, disadvantages and barriers to BIM are perceived internationally. Then, personal interviews were done
with building practitioners in Belgium. The results are summed up and compared. It appeared that many
disadvantages can be overcome by time and effort for complex projects. Nevertheless, the implementation of a BIM
still holds disadvantages and difficulties which are to be conquered, especially for smaller projects and smaller
enterprises. The paper concludes with the changing roles of building participants and the need for changing
business processes to optimally collaborate on complex building projects during its entire life cycle.
KEYWORDS: BIM, collaboration, information management, integrated design, interoperability.
1. INTRODUCTION
Over the years, many positive experiences with Building Information Modelling (BIM) and virtual modelling have
been published. In spite of these efforts, building practitioners still seem to experience barriers to the use of BIM-
structures.
The first part of the paper identifies what is intended with the term BIM, since literature defines slightly different
meanings. Due to the multiple use of the term BIM, the advantages and disadvantages of BIM are described under
three different areas which BIM supports: integrated design, interoperable project delivery, and information
management.
Also, it is clearly visible that the role of the different practitioners as well as the collaboration process is changing.
Therefore, the second part of the paper means to hint at how this change is evolving. Understanding evolving roles
can assist in optimising business and process for future building.
A literature study as well as personal interviews with practitioners are performed to answer following questions:
Which are the advantages and disadvantages of using a BIM?
What barriers to the use of BIM can be identified?
How does the use of BIM influence the collaboration process?
The interviews with practitioners were done with architects, engineers, technical advisors, constructors, and others
located in Belgium. Each of them was faced with the same questions. The main goal is to present an overview of
how BIM is seen internationally in the literature and nationally by the practitioners. In brief, the paper can be used to
identify points of attention when using a BIM.
2. BUILDING INFORMATION MODELLING IN USE
2.1 History and present day
In general, a building design is based on the wishes and performance goals as defined by the client. About twenty
years ago, the modelling was done by hand on paper and mostly in 2D. Later on, the design was drawn in CAD
programs on the computer. This CAD-use grew rapidly and transformed into 3D and finally into BIMs.
Today, a Building Information Model (BIM) can be used on several levels. A designer can implement on all the
levels or just a few. Logically, the more levels are implemented, the more benefit the design team can have from the
use of a BIM.
363
9
th

The first level contains the virtual representation of materials, constructions and assemblies in a 3D model. This
requires effort from the designer in collaboration with the constructor during early phases of the design process to
implement the acquired properties.
Using BIM on a second level can be described as creativity. Consequently, a designer can easily alter the design and
create complex structures through parametric modelling, without endangering the construction of the objects.
On a third level, a BIM is used to centralise information during all phases of a project. This way, the information is
available to all participants. Access to central information can facilitate collaboration and communication. All can
view, comment on and extract information from the model. When done consistently, conflicts between disciplines
can be solved in the early phases of the design process, leading to lower failure costs in the building process.
A possible fourth and more complex level is the simulation of building parts. For example, energy use, comfort,
daylighting or ventilation strategies can be simulated. Next, the results can be directly implemented in the BIM-
design. Finally, other parts of the building are automatically adjusted and conflicts can be identified quickly.
The fifth and last identified level is computer-driven fabrication of building elements. Computer-aided
manufacturing (CAM) facilitates the creation of complex formed components due to the fact that they are already
digitally detailed. (Kolarevic, 2003)
On the whole, a BIM can be defined as a carrier of all the information supplied by the building disciplines which are
involved during the entire life cycle of a building. The structure can hold data about function, space, materials,
demands, costs and time in a single model, but full implementation remains a challenge for the building industry.
FIG. 1: Integration and use of information in a BIM during the entire life cycle of the building.
2.2 The main reasons
Building projects have become more and more complex due to following requirements (Chen, 2005):
The limited energy use and operation cost;
The implementation of innovative techniques;
Optimisation of the indoor climate and comfort;
The difficult building sites; and
The demanded short delivery time for the building.
364
9
th

Together, these expectations lay great stress on the participants in a building design process. In order to manage and
fulfil the requirements, a changed process is required for implementation in current projects.
Moreover, the amount of building data has increased tremendously per project but also as the project grows. To keep
an overview and control, the responsible in the design and building process needs to access and control all of the
data in the BIM (see Fig. 2 and Fig. 3).
3. RESEARCH METHODOLOGY
To collect research data, in order to be able to compare with the literature findings, personal interviews were taken
from different parties involved in the building process. More specific, a software developer, a large construction
firm, an architectural firm, a project developer and a facility management firm were interviewed. All of them are
located in Belgium, but have building projects all over Europe. All of these firms have been implementing BIM-
structures for many years.
The query the participants were confronted with, was centralized around three main questions:
Which are the advantages and disadvantages of using a BIM?
What barriers to the use of BIM can be identified?
How does the use of BIM influence the collaboration process?
Other questions were posed to gauge their understanding of BIM, integrated design, operability and information
management. Generally the concepts were understood the same way, although most of the parties had a totally
different understanding of integrated design due to their professional backgrounds.
The main limitation of this study can be identified as a limited group of participants. It would be better to address
more participants from each industry. Also, a more internationally inquiry would help to see the evolutions better in
the field of practice.
4. ADVANTAGES OF BIM-USE
4.1 Literature study
4.1.1 Integrated design
A 3D virtual detailed model assists engineers and architects in visualising their technical solutions for the client.
Changes can easily be made during meetings. As such, the client understands the design more easy. Additionally,
the consultants and engineers can give feedback and advice instantaneously. The impact of demanded changes can
be calculated earlier in the design process. (Howell and Batcheler, 2005)
Because design is transformed into a virtual building, the designer is able to analyse the construction of the building
in detail and anticipate problems early in the design process (Scheer, 2005). Similarly, supporting innovative
solutions in early phases is facilitated. By complete coupling of the 3D-model with 2D-drawings, it becomes
possible to generate attuned schemes and plans with little effort in every phase of the project. (Rundell, 2007,
Schaap and Bouwman, 2006)
Sustainable building is assisted by the use of BIM, due to the transformation of the performance demands into high
technological and sustainable solutions for the clients demands. The centralised information and early
communication between multidisciplinary participants aids in creating a sustainable building. (Rundell, 2007)
4.1.2 Interoperable project delivery
For some time firms have made use of sophisticated software to assess the building physics in a project. One of the
important set-backs is the limitation of a program which can calculate only few of the variables at once. To analyse
the multiple variables questioned in the design, a BIM and all its data needs to be exchanged with different
simulation and analysis software. The data exchange is facilitated by the Industry Foundation Classes-file (IFC)
format (Zhai, 2006). The information in the model can thus be made interoperable.
365
9
th

The export of a BIM towards an IFC format makes the import into most analysis software possible. The greatest
advantage is that the advisor no longer needs to redraw the design to implement their part of the project, which saves
a lot of time and money in the process. Nevertheless, simulation is done only on a part of the building, which is
often simplified to reduce calculation times (Eastman et al, 2008). The information from IFC could therefore be too
complex and requires time to simplify the model.
In general the productivity of design and building teams will improve through sharing and communicating digital
data. Within complex projects, data management is advised to avoid mixing up versions of a model, large quantity
of models being sent around, changing of the wrong data, etc.
Long term relationships between parties in the process can be achieved, so consultants stay involved early in the
process (Howard and Bjrk, 2008). By working together on several projects, the participants achieve knowledge and
experience from each other. Taking long term collaboration a step further, these partners can be brought together
under a corporate umbrella. This way they can provide more services to the owner, for example operating and
maintaining the building by using the BIM for the entire life cycle and thus managing the total cost of ownership
(Jordani, 2008). Also, work by one member of the project team could benefit another. This implies that benefits
ought to be shared by all involved in the entire process (Howard and Bjrk, 2008).

FIG. 2: Amount of available information during the design and building process in a traditional building process.
(Kolarevic, 2003)

FIG. 3: Amount of available information during the design and building process by using a BIM.
366
9
th

4.1.3 Information management
Decisions are made earlier in an integrated process and this implies time-savings by parallel work (Howard and
Bjrk, 2008). BIM facilitates the fabrication of building elements with CAM processes since the digital information
is already available and can be extracted from the model indisputable. (Scheer, 2005, Avoort, 2007)
4.2 Personal interviews
Architects indicated the facilitated combination of disciplines as the greatest advantage of applying a BIM in the
design and building process. Moreover, implementation of technical advice in early phases of the design is pointed
out as crucial to avoid extra costs during construction.
Drawing up the model quickly and extracting plans, tables and cost-calculation were defined as the greatest
advantages by the architects, designers, but also by a constructor.
The resistance of clients and designers for integration of sustainable solutions in early design phases has diminished
over the last decade. This implies that more players have evolved towards BIM-use and are now in competition,
which is undoubtedly beneficial for the client.
In all the personal interviews it was corroborated that long-term collaboration works advantageous for the growth of
knowledge and experience of all participants.
Especially the designers added that international collaboration becomes possible with BIM, also improving
knowledge and experience on complex and sustainable buildings.
By managing all data centrally, it is found to be easier and faster to control and recover required information by all
interviewees.
5. DISADVANTAGES OF BIM-USE
Limited creativity, especially in early design phases can be seen as a disadvantage. Designers require large
flexibility in design while maintaining an intelligent BIM. This is a paradox, for intelligence exists of rules for an
element while flexibility exists of freedom to transform the element. (Scheer, 2005)
Each participant in a building project chooses the software that fits his/her work best. This implies the use of several
tools in the design process. Interfaces between tools gain in importance for there is no existing software that meets
all the demands of the project (Hensen and Radoevi, 2004, Franken, 2005). Hence, the difficulty of implementing
the system - including software, tools, platforms, networks, etc. - into the business-processes remains a burden on
the tasks and responsibilities of the participants. (Schaap and Bouwman, 2006)
Most benefits of BIM imply long term relationships. In Belgium, there are many ways and contractual forms to
work on a project, but long term relationships are not easily formed. Accordingly, the client requires knowledge of
the possible contracts and decides on the form of collaboration with the help of his advisors. Some forms allow
participants to work together on a series of projects by the use of framework agreements and others support
collaborations throughout the life cycle of the building by including operation and maintenance in the contract.
(Howard and Bjrk, 2008)
367
9
th

The method of export and import of BIMs is highly anticipated, but still has several disadvantages like: time
consuming preparation, incomplete exports and required customisation, complex and unmanageable files, the
needed change management, little available information about possibilities, etc. (Universit de Lige, 2006). Open
standards can be a solution to this problem, but then the requirements for exchange and copyright must be defined in
the contract.
Clear agreements on the way of communication and collaboration are required from the first initiative to avoid
personal gain or interest of the participants (Schaap and Bouwman, 2006). A common goal, shared responsibility
and shared profit are thus important features of attention for the client.
Responsibility of the information in the model is a point of attention. It should be clearly stated who in a project is
responsible for the gathering, delivering, maintaining and controlling the information flow into and from the BIM
(Scheer, 2005). After the delivery of the project the question arises who will update the model during operating and
maintenance phases.
To be able to draw a model quickly, databases are crucial. When not made up before the start of the modelling, a
large amount of time is required to set the elements right. Since a standard interface for extracting tables and cost-
calculation does not exist internationally, each company works with its own, local tools. Much research is being
performed on this subject and it is expected that the local systems are soon to be internationally useable.
Contracts and agreements are made up standard. Integrated design with a BIM requires a total change of thinking
before drawing up the contracts. This is difficult for the client as he/she often has little knowledge nor experience.
Therefore, the architect identifies to be the pre-eminent participant to advise and support this part of the process.
To control the amount of data that is sent around and maintain an overview, a data manager is to be assigned for
complex projects. This person has knowledge on the information flows and controls the data in the BIM. As a result,
this is a completely new function and holds great responsibility, which implies a difficulty in the start of a project.
Many interviewees name the need for such an information manager, but few have assigned one in past projects.
6. BARRIERS TO USING BIM
The learning curve for the BIM-tools can be identified as a clear barrier. The employees working on a project all
need training with the software and the multidisciplinary capabilities. This is a large investment in human resources
for a building project. Therefore, a conversion of the companys mentality is required towards working with BIM as
an advantage the future. Also, the skills need to be managed and updated, since the functionalities of BIM-tools are
evolving rapidly. (Eastman et al, 2008) Not only the personnel, but also the top of the company needs to be involved
in the process of change towards BIM-use, and need be convinced of the benefits for the business in the future.
(Laverman, 2007)
Other barriers are the limitation and complexity of export and exchange of models and satisfaction to traditional
methods (Yan and Damian, 2008).
Both the designers as the constructors identified the costs of BIM-software too high for a single large project. Unless
the client explicitly demands a BIM, the motivation for BIM-use is quite low. For smaller project the use of BIM
appears not to be profitable. Also, the learning-curve and motivation of individuals in the firm is seen as time and
cost consuming, since the people working with the BIM are required to change their way of working. From
368
9
th

traditionally solving problems towards anticipating difficulties in a 3D-model in early design phases is a process that
demands management and time. Changing people demands effort and time. (Laverman, 2007)
Clients apparently do not ask for a BIM which holds all the information needed for the facility management. They
often ask for text documents or calculations, but not for a 3D model of the building. A reason for this could lie in the
incapability of the facility manager to read and interpret the information that lies in a BIM.
The constructors identified in the interviews the importance of involving the management board of the business at
the start of BIM-use. Without the support of the board, it seems almost impossible to integrate the use of BIM for a
project, let alone throughout the entire concern.
In general, constructors (especially the small companies) indicate their satisfaction with traditional methods. They
often have worked in a traditional way for years and feel hesitant to change. Also, they do not work on complex
projects with a large amount of information. On the other hand, many of the larger constructors did feel the need for
change and are implementing the software and the collaboration process on several levels.
7. INFLUENCE OF BIM ON THE COLLABORATION PROCESS
The use of the model and the contained information requires information management, especially in early design
phases, where large quantities of data are produced concurrently. This additional task is best assigned to a party with
the required knowledge and experience.
Communication is clearest when assisted by 3D models, since each participant has his own technical background
which complicates communication. Owing to change from 2D to 3D, investments in software-tools are required.
Also, all participants need to know how to work with the software and understand the importance of working
towards the same goals. They are to be identified as partners in a team, which is frequently experienced as a shift in
collaboration and thinking.
From the personal interviews appeared that not only a changing organisation of the teams for collaboration on one
project, but also long-term commitment for collaboration on multiple projects. The architects noted a change in
thinking along with the technical engineers, more specifically the changing mentality from solving a difficulty in the
design towards avoiding and anticipating conflicts in early design.
Since complex projects require decisions on technical matters, the architect has gained an even more important role
as advisor of the client to help him/her understand the concept and the context. In the same way, other advisors are
necessary to assist the owner in making decisions from early design till the dismantling of the building.
8. CONCLUSIONS
From the personal interviews, it can be seen that the national view on BIM is quite the same as internationally found
in the literature. Often 3D models are used on smaller scale and fewer levels.
Despite the disadvantages and barriers that can be found still, the use of BIM has increased enormously. This clearly
influenced the collaboration process.
Clients of large projects appear to be more and more aware of the advantages for their buildings as well as their
profit in the life cycle of these buildings.
As implementation of a BIM cannot be expected to go over night, the stakeholders in a project need time and some
guidance to adapt their way of thinking as well as their way of working. Both the modelling-software as well as the
collaboration are required to make the process profitable.
Further research into the existing roles of participants in a complex building project is necessary to identify the
evolution towards the collaboration and communication in building projects for the future. Since the designers and
the constructors identified the support of the board for BIM-implementation as highly important, it can be defined
that the implementation of BIM in the entire industry requires a new way of thinking.
Concluding, it can be stated that disadvantages of and resistance to BIM are diminishing, while the experience with
BIM in the building industry is rising. The impacts on energy importance, techniques, and buildings on the whole
cannot go unnoticed.
369
9
th

9. REFERENCES
Avoort P.V.d. (2007). Investeren in automatisatie stelt onze toekomst veilig, Schrijnwerk, No. 121, 34-35.
Chen S.-J.G. (2005). An Integrated Methodological Framework for Project Task Coordination and Team
Organization in Concurrent Engineering, Concurrent Engineering: Research and Applications, Vol. 13, No.
3, 185-198.
Eastman C., Eastman C.M., Teicholz P., Sacks R. and Liston K. (2008). BIM Tools and Parametric Modeling, BIM
Handbook: A Guide to Building Information Modeling for Owners, Managers, Designers, Engineers and
Contractors, John Wiley and Sons, Hoboken, USA.
Franken B. (2005). Real as data, Architecture in the digital age: design and manufacturing, (Kolarevic B., editor),
Taylor & Francis, New York & London, 121-138.
Hensen J.L.M. and Radoevi M. (2004). Teaching building performance simulation - some quality assurance issues
and experiences, Proceedings of the 21st PLEA international conference Passive and low energy
architecture, Eindhoven, The Netherlands, 1209-1214.
Howard R. and Bjrk B.-C. (2008). Building information modelling - Experts' views on standardisation and industry
deployment, Advanced Engineering Informatics, Vol. 22, No. 2, 271-280.
Howell I. and Batcheler B. (2005). Building Information Modeling Two Years Later - Huge Potential, Some
Success and Several Limitations, The Laiserin Letter, University of Utah, USA, 1-9.
Jordani D. (2008). BIM: A Healthy Disruption to a Fragmented and Broken Process, Journal of Building
Information Modeling, Vol. 2, 24-26.
Kolarevic B. (2003). Architecture in the digital age: design and manufacturing, Spon Press, New York & London.
Laverman W. (2007). BIM, bir, bom, het Bouwwerk Informatie Model komt er aan!, Building Innovation, March,
40-41.
Rundell R. (2007). 1-2-3 Revit: BIM and Analysis for Sustainable Design, Cadalyst, April 5.
Schaap H.A. and Bouwman J.W. (2006). Toekomst voor het bouwproces; een 3D-benadering, CUR-report 218,
(COINS, editor), Gouda, The Netherlands, 1-39.
Scheer D.R. (2005). Building Information Modeling: What About Architecture?, The Laiserin Letter, University of
Utah, USA, 1-6.
ULg. (2006), System simulation in buildings, Proceedings of the 7th international conference, Les editions de
lUniversit de Lige, Belgium.
Yan H. and Damian P. (2008). Benefits and Barriers of Building Information Modelling, Proceedings of the 12th
International Conference on Computing in Civil and Building Engineering, Beijing, China.
Zhai J.Z. (2006). Application of Computational Fluid Dynamics in Building Design: Aspects and Trends, Indoor
and Built Environment, Vol. 15, No. 4, 305-313.
370
9
th

THE INTRODUCTION OF BUILDING INFORMATION MODELLING IN
CONSTRUCTION PROJECTS: AN IT INNOVATION PERSPECTIVE
Arjen Adriaanse, Dr.
University of Twente/ Ballast Nedam, the Netherlands
a.m.adriaanse@utwente.nl; www.cme.ctw.utwente.nl/ www.ballast-nedam.nl
Geert Dewulf, Prof.dr.
University of Twente, the Netherlands
g.p.m.r.dewulf@ctw.utwente.nl; www.cme.ctw.utwente.nl
Hans Voordijk, Dr.
University of Twente, the Netherlands
j.t.voordijk@ctw.utwente.nl; www.cme.ctw.utwente.nl
ABSTRACT: Limited research has been devoted to the introduction and use of BIM across organisational
boundaries in construction projects. The objective of this research is to develop guidelines for a successful
introduction of these BIM technologies. We approach the introduction and use of BIM from an IT innovation
perspective. Because of the explorative nature of the research the introduction and use of BIM will be analysed in-
depth in a construction project. We draw on the principles of action research to apply, validate, and refine our
theoretical concepts. Our research offers two contributions. First, we document and analyse in-depth the
introduction of BIM across organisational boundaries in a construction project. Second, we develop, validate, and
refine guidelines for the introduction of BIM.
KEYWORDS: BIM, construction projects, IT innovation, action research.
1. INTRODUCTION
Building Information Modelling (BIM) can offer many benefits in improving interorganisational communication,
cooperation, and coordination in construction projects. These object-oriented models can be exchanged between
organisations, merged into one model and used, for example, for virtual design reviews, automated quantity take-
offs, and virtual constructability reviews. However, the use and exchange of BIM across organisational boundaries
in construction projects is still limited and not as effective and efficient as it could be.
Numerous scholars have discussed the opportunities and potential benefits of the use of BIM (McKinney and
Fischer, 1998; Whyte et al., 2000; Akinci et al., 2002; Bouchlaghem et al., 2005). In addition, several investigators
documented and analysed the use of these applications in real time construction projects (Bouchlaghem et al., 2005;
Harty, 2005). However, limited research has been devoted to the introduction and use of BIM across organisational
boundaries in construction projects. There is clearly a need for more understanding of these processes so that the
potential benefits of BIM in the future can be realised. The objective of this research is to develop guidelines for the
successful introduction of BIM across organisations in construction projects.
We approach the introduction and use of BIM from an IT innovation perspective (e.g., Cooper and Zmud, 1990;
Swanson, 1994; Swanson and Ramiller, 2004). The IT innovation perspective considers the introduction and use of
BIM as a social instead of a technical phenomenon and enables researchers to analyse its introduction and use in its
social and interorganisational context. It focuses on conditions that need to be secured for successful introduction
and use of BIM. Because of the explorative nature of this research the introduction and use of BIM will be analysed
in-depth in a construction project. We draw on the principles of action research (Baskerville and Wood-Harper,
1996; Baskerville, 1999) and intervene in situations in order to apply, validate, and refine our theoretical concepts.
Our research offers two contributions. First, we document and analyse in-depth the introduction of BIM across
organisational boundaries in a construction project. Second, we develop, validate, and refine guidelines for the
introduction of BIM.
The paper is organised as follows. First, we review the IT innovation processes. Second, the research design is
presented. Third, the results of the case study are summarised. The final part contains conclusions.
371
9
th

2. THE IT INNOVATION PROCES
In this paper, we use an IT innovation perspective. Swanson and Ramiller (2004, p.556) define an IT innovation as the
process by which IT comes to be applied in novel ways. IT innovation may be analysed from the perspective of an
organisations subunit, an organisation, or even different organisations (Swanson, 1994). In case of the introduction of
BIM in construction projects to support interorganisational communication, cooperation, and coordination an
interorganisational IT innovation perspective becomes important (Harty, 2005). IT innovation is often viewed as a
process within which various stages may be distinguished: comprehension, adoption, implementation, and assimilation
(Swanson and Ramiller, 2004; see also Cooper and Zmud, 1990). The stages and the considerations and decisions
that need to be made in each of these stages are discussed below from an organisation perspective. In the case study
these theoretical concepts will be translated to the interorganisational context in a construction project and applied to
manage the introduction of BIM.
2.1 Comprehension stage
In the comprehension stage, individuals or other decision-making units actively or passively gather and evaluate
information and scan (1) organisational challenges and opportunities, and (2) IT solutions and its benefits to find a match
between IT solutions and the firms own circumstances (Cooper and Zmud, 1990; Rogers, 2003). Through the
sensemaking efforts they learn more about the IT innovation and develop an attitude or stance towards it (Swanson and
Ramiller, 2004). Sensemaking activities can be steered by either organisational needs (pull), technological innovation
(push), or both (Cooper and Zmud, 1990). Based on the sensemaking activities they may decide to become a prospective
adopter.
According to Swanson and Ramiller (2004, p.561) innovators should not take generalised claims about the innovations
benefits and applicability at face value but will instead critically examine their local validity. In addition, they should
create situations for rich and context-specific learning to facilitate organisational sensemaking (e.g., demonstrations, site
visits, experimental prototyping) (ibid.).
2.2 Adoption stage
After a prospective adoption decision a deeper consideration of the IT innovation follows in the adoption stage. Both the
business value of the IT innovation and the challenges presented by the prospective change are likely to be weighed
before the organisation decides whether to proceed and commit resources to the innovation (Swanson and Ramiller,
2004). This stage ends with a decision to adopt, not to adopt (i.e., reject) or to defer the adoption of the IT innovation.
The decision for implementing the IT innovation can be based on rational and political negations (Cooper and Zmud,
1990). An adoption decision may be voluntary and based on own considerations. However, an adoption decision may
also be mandated by other firms or by the management of the own firm (Rogers, 2003; Adriaanse, 2007). Rogers (2003)
calls this phenomenon a mandate for adoption. In these situations there is no other choice than to decide to adopt the IT
innovation.
Swanson and Ramiller (2004, p.562) state that a rational in favour of adopting should be context-specific, rich in its
consideration of local organisational facts, and focussed on the innovations potential contribution to the firms
distinctive competence. This rationale may point against adoption, or it may favour deferred adoption (ibid.). Sometimes
a firm may decide to experiment or pilot with the IT innovation first before making an adoption decision (Rogers, 2003;
Swanson and Ramiller, 2004). In this view, a firm is implementing parts of the IT innovation before adopting it.
Adriaanse (2007) developed a theoretical model containing categories and subcategories that influence the way actors
use interorganisational IT in construction projects. These categories and subcategories are defined in Table 1. In the
adoption stage, the categories and subcategories can help people responsible for implementing an IT innovation to
identify technical and nontechnical risks and barriers related to the introduction and use of IT in a specific project.
Based on this analysis, they can formulate and implement measures and solutions to control risks and reduce or
eliminate barriers. If barriers cannot be eliminated or reduced enough or if risks cannot be mitigated to acceptable
levels they should not use IT or limit the scope of the use of IT (e.g., limit the scope to only some organisations,
only some aspects of the IT application or only some parts the construction project).
372
9
th

Table 1: Categories and subcategories influencing the use of interorganisational IT.
Category, subcategory Definition
1. Personal motivation The extent to which actors are willing to use interorganisational IT themselves. Personal
motivation influences both the willingness of the actors to use IT and their willingness to
invest resources to overcome barriers to the intended use of IT.
1a. Perceived benefits and
disadvantages of IT use
The extent to which actors perceive the use of IT as benefiting and/or disadvantaging them.
1b. Perceived time pressure The extent to which actors perceive that they have to act quickly when using, or considering
the use of, IT. A high level of perceived time pressure can moderate personal motivation
because of the highly perceived benefits of the use of IT.
2. External motivation The extent to which actors are forced by other actors to use IT. External motivation influences
both the use of IT and the efforts made to invest time and money to overcome barriers to the
intended use of IT.
2a. Availability of contractual
arrangements about IT use
The extent to which actors are forced to use IT or other means of communication because this
is mandated in the contract.
2b. Presence of a requesting actor The extent to which another actor requests certain action(s) (e.g. use of IT, or non-use of IT)
to take place and the extent that this request impacts on actors.
3. Knowledge and skills to use IT The extent to which actors know how to use IT. When knowledge and skills are limited, the
actors themselves are the ones restricting the use of IT.
3a. Clarity of procedural agreements The extent to which actors know how to act concerning the IT application (e.g., what
information has to be communicated to whom, and in what form and at what time) and these
actions support the intended use of IT.
3b. Clarity about the operation of IT The extent to which actors know how to operate the application.
4. Acting opportunities The extent to which actors are able to use IT in the intended way. When the acting
opportunities are limited, IT is not able to support the actions of the actors involved.
4a. Alignment between IT and
working practices
The extent to which IT fits in with actors working practices in the project and their
organisation(s).
4b. Availability of technical means The extent to which technological aspects restrict actors in using IT in the intended way.
2.3 Implementation stage
Implementation is the critical gateway between the decision to adopt the IT innovation and the routine use of the
innovation within an organisation (Klein and Sorra, 1996). In the implementation stage the IT innovation is developed
and/or tailored to the firm-specific context, installed, and maintained, the organisational procedures are revised and
developed, organisational members are trained both in the new procedures and the IT application (i.e., adaptation)
(Cooper and Zmud, 1990). In addition, organisational members are starting to use the IT application and are induced to
commit to IT application usage (ibid.). Adaptation of the IT innovation may continue based on users experience. At the
end of the implementation stage the IT innovation is implemented and users are committed to its use.
Until the implementation stage the IT innovation is a mental exercise of thinking and deciding. Now the IT innovation is
actually put in practice and its use will expand along the way (Rogers, 2003; Swanson and Ramiller, 2004). According to
Swanson and Ramiller (2004, p.562) innovators should be attentive to problems of all kinds, treating them not merely as
obstacles to be overcome but also as potential symptoms of prior misconceptions. They should be sensitive to small
oversights or areas of neglect that might lead to larger failures (ibid.).
373
9
th

2.4 Assimilation stage
In the assimilation stage, the IT innovation diffuses across the organisation and becomes routinised in the activities of the
organisation (Purvis et al. 2001). It becomes to be absorbed into the work life of the firm and demonstrates its usefulness
(Swanson and Ramiller, 2004). The use of the IT innovation is seen as a normal activity and the IT innovation is used to
its fullest potential (Cooper and Zmud, 2000). According to Swanson and Ramiller (2004, p.563) innovators should
remain open to surprises, continued learning, and the potential for adaptations that address unanticipated problems or
realise unforeseen potential.
3. RESEARCH DESIGN
The introduction and use of BIM (in this case 3D/4D modelling; 4D = 3D + time) across organisational boundaries will
be analysed from an IT innovation perspective in-depth in a power plant design and construction project. The project is
located in the Netherlands. The civil, structural, and architectural (CSA) work is awarded to the design and construction
firm in December 2007. The amount tendered for the CSA work is about 170 million, and the project needs to be
finished in 2011. For one part of the project (Pack A: concrete foundations, steel constructions, and cladding of the
power plant) the design and construction firm is only responsible for construction. For another part (Pack B: cooling
water circuit) the design and construction firm is responsible for both design and construction.
As will be described in the next section the design and construction firm introduced 3D/4D modelling in the project. The
engineering company (business unit of the design and construction firm) will make 3D/4D models and will distribute
these to the contractor (also a business unit). They will use 3D/4D models for multidisciplinary design and
constructability reviews and to support communication with the client and subcontractors. One subcontractor is able to
make a 3D model of the steel constructions and will distribute this model to the design and construction firm. The
engineering company merges this model with the other models. Participating organisations are shown in Figure 1. 3D/4D
models are used to support communication with the client but are not distributed to the client.
FIG. 1: Organisations using and exchanging 3D/4D models.
In this research, we draw on the principles of action research (Baskerville and Wood-Harper, 1996; Baskerville, 1999)
and observe events for a period of time, in order to try to get as close as possible to the people being researched. We try
to understand social situations from the standpoint of the participants involved and intervene in situations in order to
apply, validate, and refine our theoretical concepts. More in particular, we try to understand why actors make certain
decisions in the comprehension, adoption, implementation, and assimilation stages and explore how these decisions can
be influenced to manage the introduction of BIM successfully. Therefore, we evaluate the value and usefulness of the
theoretical guidelines.
During the case study, a researcher actively participated in the research and collected data using multiple techniques.
First, the researcher spent most of the time observing participants and informally talking to them. Participant observation
took place during the daily routine and in meetings. The researcher had complete access to all internal project meetings
within the engineering company and the contractor. Second, the researcher conducted many informal and semi-
structured interviews to capture participants perceptions and understanding. The researcher tried to see the world from
the participants point of view. Finally, the researcher examined documents such as contract documents, minutes of
meetings, and procedures from the quality management system. The researcher followed the project for 7 Months
(December 2007 June 2008).
374
9
th

4. RESEARCH RESULTS
This section presents the introduction process of 3D/4D modelling in the construction project from an IT innovation
perspective. At the end of each stage of the IT innovation process, the findings will be compared with the theoretical IT
innovation process as described in Section 2 to show differences, similarities, and refinements. Because of space
limitations, we will only present the highlights of each stage without trying to be exhaustive.
4.1 Comprehension stage
Immediately after the contract was awarded to the contractor in December 2007 one of its work planners contacted the
BIM Program Manager
1
. He saw some important benefits of 3D/4D modelling in the project. Several weeks ago he
attended an information meeting about 3D/4D modelling in which a software vendor showed 3D/4D opportunities and
benefits for construction projects. As a result, in general terms the work planner knew about 3D/4D technologies and its
potential benefits. The BIM Program Manager was able to help the work planner to translate the project challenges to
3D/4D opportunities and its benefits based on experiences in using 3D/4D technologies in two tender projects. Through
these sensemaking activities they could give general claims about 3D/4D opportunities and benefits local validity.
In a follow up meeting they invited a drafter/modeller of the engineering company as well to discuss the 3D/4D
opportunities in the project. The modeller had experiences with 3D/4D modelling and extracting 2D form drawings and
quantities from the 3D model in two tender projects. Based on the discussion they expected that the project would greatly
benefit from 3D/4D modelling (i.e., making 3D and 4D models (including cranes, other critical equipment, and
temporary construction elements) and extracting 2D form drawings and quantities from it). In their view, the main
benefits were:
Provide better insight into the power plant earlier in the process: 3D models provide better insight to the
contractors work planners and works foremen, the client, and subcontractors. This reduces misunderstandings.
Better insight in the construction process: 4D models provide better insight in who is going to do what at what
moment in time. This insight improves coordination between disciplines and organisations.
Easier quantity take-off: 3D models can be used for efficient quantity take-offs (e.g., surface areas, volumes).
This is especially interesting for complex parts of the foundations.
Optimisation of work methods: work planners and works foremen can use 3D/4D models to detect spatial
conflicts in the construction process, show alternatives, and in the end optimise work methods.
Together they discussed the risks that are associated with the introduction of 3D/4D modelling in the project and
measures to mitigate these risks as well. These risks and measures are shown in Table 2.
Table 2: Risks and measures to mitigate these risks.
Risks Measures
A limited number of drafters that is able to
model (i.e., only one experienced
drafter/modeller is available in the
engineering company).
Limited experience with 3D/4D modelling.
A limited amount of time to learn 3D/4D
modelling in the project (tight schedule).
Two drafters will follow a Revit Structure course. One of these drafters can
model in the project together with the experienced drafter/modeller.
A fallback option to traditional AutoCad will be available.
Limit the scope first to activities the experienced modeller has done before:
(a) Pack A: 3D/4D models with a limited level of detail (concrete
foundations, buildings), and (b) Pack B: 3D/4D models of the cooling water
circuit, form drawings (detail and reinforcement drawings are excluded),
and quantities.
Together they prepared a presentation for the CEO of the contractor, the project management of the project and some
other members of the project team (contractor and engineering company). Several days later (December 19 2007) they
gave the presentation. They presented the 3D/4D opportunities, the specific benefits for the project, the 3D/4D modelling
activities, the required time investment associated to these, the risks of 3D/4D modelling, and measures to mitigate these
risks. During the presentation there was a lively and enthusiastic discussion between the attendees. Attendees agreed on

1
The BIM Program Manager is responsible for implementing BIM into the design and construction firm.
375
9
th

the benefits and saw some additional opportunities. One mentioned the opportunity to extract m
2
formwork from the 3D
model. Another mentioned the opportunity to exchange models with subcontractors. Maybe the subcontractors will be
able to provide 3D models. According to the CEO of the contractor 3D/4D activities should focus on optimising work
methods. The models should be used to support communication between, for example, designers, work planners, and
works foremen to optimise the construction process. This was most important to him.
Three persons had to make the decision to implement 3D/4D modelling in the project: the Project Director, the Design
Manager, and the CEO of the contractor. Before the start of the presentation they discussed the introduction of 3D/4D
modelling in the project. They decided not to implement 3D/4D modelling. The Project Director had had some bad
experiences with using these kinds of technologies in another project. The Design Manager was positive about 3D/4D
technology and had followed the introduction and use of 3D/4D modelling in another project (a tender). However, he
was not enthusiastic about 3D/4D modelling in this project because of the lack of a fallback option and the limited
number of experienced modellers. The CEO of the contractor stressed: We are not going to experiment in this project!
Time pressure is high and stakes are big.
During the presentation they changed their minds. Especially the examples from another project (a tender), the concrete
benefits for this project, the limited time investment needed, and the controllability of the risks resulted in enough
confidence to give it a try. Because the CEO promised to allocate budget, if needed, for 3D/4D modelling in the project
and the BIM Program Manager was willing to manage the implementation process they decided to implement 3D/4D
modelling in the project. However, first they requested a thorough implementation plan that should be finished 15
January 2009. The BIM Program Manager proposed to write this plan.
The steps taken in the project to successfully complete the comprehension stage are shown in Figure 2. First, the BIM
Program Manager, work planner, and drafter/modeller defined the added value for the specific project based on a 3D/4D
information meeting, their experiences with 3D/4D modelling in tender projects, and their understanding about the
project challenges. Second, they analysed the implementability by assessing the risks of implementing 3D/4D modelling
and measures to mitigate these risks. Third, the BIM Program Manager was willing to manage the 3D/4D adoption (and
implementation) process. Fourth, these three aspects created management support and finally convinced the decision-
makers to make a positive prospective adoption decision. However, a thorough implementation plan (see adoption stage)
needed to be written first before a definite implementation decision would be made. The first step is in line with the
important sensemaking activities as discussed in the theoretical comprehension stage in Section 2. However, the other
steps are lacking. These appeared to be essential to create management support and to successfully complete this stage.

FIG. 2: Steps of the comprehension stage.
4.2 Adoption stage
After the go decision several meetings took place in which the BIM Program Manager, the work planner, the
experienced drafter/modeller, the Design Manager, and the Head of the Drafting Department (1) worked out the
innovation idea (activities and associated changes in working practices), (2) analysed the implementability of the idea,
376
9
th

and (3) discussed parts of the implementation plan. Input for these discussions were the insights, and decisions made in
the comprehension stage. The BIM Program Manager managed this process. The results of the discussions of this
adoption group are presented below.
4.2.1 Work out the idea: Analysis of activities and changes in working practices
The engineering company will be responsible for 3D/4D modelling. Two drafters/modellers will make the models. The
drafters/modellers have to execute the following general activities: (1) create an object-oriented 3D model, including
large cranes, other critical equipment, and temporary construction elements (3D modelling), (2) create electronic
drawings sheets from the 3D model and add details and annotations to them (drafting; form and detail drawings), (3) link
objects in the 3D model with tasks in the project schedule (4D modelling), (4) extract quantities from the 3D model
(quantity take-off), and (5) merge the models for visualisation. The drafters/modellers will use Revit Structure (3D
modelling, drafting, quantity take-off) and NavisWorks (4D modelling) for executing these activities. The group further
worked out the activities and defined the input and output of these activities. The experienced drafter/modeller is
available for the project. He is experienced in 3D and 4D modelling, extracting form drawings, and doing the quantity
take-off in tender projects. Making detail drawings is a new activity for him. The other drafter/modeller that needs to be
added to the project team is not experienced at all in 3D/4D modelling.
Before the drafters/modellers communicate information formally to the contractor this information needs to be approved
by the Design Manager. Drawings will still be communicated in paper-based form to the contractor. These drawings
instead of the digital model take precedence. The paper drawings will still follow the traditional approval and distribution
processes within the engineering company. The 3D/4D models will not have formal status.
The contractor will use the 3D/4D models for reviews and may decide to use them in meetings with the client and
subcontractors. The project team can use viewers or can ask the drafters/modellers to show aspects in the models. The
use of 3D/4D models and viewers is new to all the contractors participants involved.
4.2.2 Analyse the implementability of changes
The BIM Program Manager used the categories and subcategories from Table 1 to analyse the implementability of
the changes for each actor (i.e., experienced and unexperienced drafters, viewer users). He analysed (1) potential
drivers, barriers, and risks, and (2) solutions and measures for these. He refined the risks from the comprehension
stage (see former subsection) based on his increased understanding in the adoption stage about 3D/4D modelling
activities and the associated changes. For both modellers/drafters some results of this analysis are presented in Table
3. The results are grouped to the four theoretical categories.
Table 3: Analysis of implementability of changes for the drafters/modellers.
Analysis (drivers, barriers, risks) Solutions/mitigating measures
Personal motivation
The experienced drafter/modeller is highly
motivated to use 3D/4D technology and to support
the unexperienced drafter/modeller in learning to
use this technology.
The unexperienced drafter/modeller is critical
towards some aspects of 3D/4D technology.
Limit the scope of 3D/4D modelling activities first. Increase the
scope only if the initial scope works well.
The experienced drafter/modeller will show opportunities and
benefits of 3D/4D technology to the unexperienced drafter.
External motivation
Project management allows the use of 3D/4D
technology. However, this support will disappear if
the project management becomes disappointed in
the technology or if confidence in the use of the
technology disappears.
The project team may disturb the drafters/modellers
to show things in the models. This limits their
drafting productivity.
No other (experienced) drafters/modellers are
available.
Show the first positive results of 3D/4D modelling as soon as
possible to the project management and team.
Limit the scope of 3D/4D modelling activities first. Increase the
scope only if the initial scope works well (management support
is needed for that).
Drafting has priority over other modelling activities.
Viewer will be available to the project team. They can use the
viewer to analyse 3D/4D models themselves.
A fallback option to traditional AutoCad needs to be available.
However, drafters/modellers will not be encouraged to use this
377
9
th

Analysis (drivers, barriers, risks) Solutions/mitigating measures
option.
Two drafters will follow a Revit Structure course. One of these
drafters can model in the project.
Knowledge and skills
The experienced drafter has no experiences with
detail drawings.
The unexperienced drafter has no experiences with
3D/4D modelling.
Time pressure is high. It will be difficult to invest
time to learn to use 3D/4D technology.
The scope of 3D/4D modelling is limited first to limit the
number of new aspects. Increase the scope only if the initial
scope works well.
User support will be provided by a software vendor (if needed).
Two drafters will follow a Revit Structure course. One of these
drafters will model in the project.
The experienced drafter will support the unexperienced drafter
in using Revit Structure.
A fallback option to traditional AutoCad needs to be available.
However, drafters/modellers will not be encouraged to use this
option.
Acting opportunities
Unclear version management, approval, and
distribution processes for 3D/4D modelling.
The Design Manager and BIM Program Manager will develop
new 3D/4D processes and procedures.
4.2.3 Develop the implementation plan
After a while, enough information was available to move from analysis to action. The BIM Program Manager
started to think about the implementation strategy and implementation plan. Because the drafters/modellers had to
start almost immediately (i.e., limited time to prepare), the high time pressure (i.e., limited time to learn), the big
stakes, and to ensure the controllability of the implementation process the BIM Program Manager decided together
with the adoption team to choose an incremental (i.e., step-by-step) implementation strategy. This limited the risks
of the introduction and use of 3D/4D modelling and increased the chance to match expectations. There was no
management support for taking risks. The following decisions were being made based on the incremental
implementation strategy:
Pack A (power plant): the experienced drafter/modeller will start with modelling Pack A (i.e., concrete
foundations and buildings). Because the design and construction firm is only responsible for construction
the modelling activities will be limited to general forms (i.e., a limited level of detail). These modelling
activities will take some days. Steelwork and cladding will be executed by subcontractors. These
subcontractors are not yet selected. If these subcontractors are selected and able to provide 3D models and
if other 3D/4D modelling activities are successful the possibilities of exchanging models with
subcontractors will be further considered.
Pack B (cooling water circuit): after the unexperienced modeller has followed his training course the
modellers will start with Pack B. The contractor is responsible for both design and construction of the
cooling water circuit. Therefore, the design will be executed in house by the engineering company. The
engineering company will start with designing the cooling water intake. The adoption group decided to
model the cooling water intake first. The cooling water outfall will only be modelled if modelling of the
cooling water intake goes well and if there is enough capacity to model the outfall. In addition, the
engineering company will start with extracting form drawings from the 3D model. Only if this proves to go
well detail drawings (and maybe reinforcement drawings) will be extracted from the models as well.
Based on these decisions the BIM Program Manager started with writing the project plan. This plan described what
will be done, by whom, when, and how. In the implementation plan the added value of 3D/4D modelling, the 3D/4D
modelling activities (and their input and output), the associated investment, and the main implementation
interventions were described.
January 15 2008 the BIM Program Manager sent the implementation plan for approval to the Project Director. The
Project Director approved the plan. According to the Project Director his 3D/4D knowledge was too limited to
378
9
th

assess the plan in a proper way. However, the implementation plan was in line with the given presentation in the
comprehension stage. This was most important to him. However, he would critically follow if 3D/4D modelling
activities match expectations and would terminate the activities if this is not the case. It was most important to him
that drawings will be delivered in time.
The steps taken in the project to successfully complete the adoption stage are shown in Figure 3. First the adoption
team worked out the idea (activities, changes). Second, the BIM Program Manager analysed the implementability of the
changes by analysing for each actor (a) drivers, barriers, and risks, and (b) solutions to barriers, and measures to mitigate
risks. The theoretical categories and subcategories were useful. They had been used as a checklist to ensure that all
relevant context-specific considerations were made. Third, the BIM Program Manager developed the implementation
plan in which the results of the former steps were incorporated and in which he described concrete activities for the
implementation and assimilation stages. He based his plan on an incremental implementation strategy to limit the risks
of the introduction and use of 3D/4D modelling and to get management support. Finally, this plan created the needed
management support and resulted in an adoption decision by the Project Director. Each of these steps appeared to be
essential to work out the IT innovation, its business value, the challenges presented by its prospective change, and the
implementation plan. These steps together form a context-specific rational in favour of adopting 3D/4D modelling in the
project. In fact these steps are a further refinement of the general aspects of the theoretical adoption stage as discussed
in Section 2.
FIG. 3: Steps of the adoption stage.
4.3 Implementation and assimilation stages
After the Project Director approved the implementation plan the BIM Program Manager started with executing
activities as described in the implementation plan. Examples of activities are (a) developing procedures for
approving and distributing 3D/4D models, drawings, and quantities, (b) training and support for the
drafters/modellers, and (c) installation of model viewers. The experienced drafter/modeller gave instruction about
the use of the viewer to the project participants that were interested to use the models.
At the end of January, the experienced drafter/modeller started with making a 3D model of the power plant (Pack
A). He modelled the power plant in several days based on the general 2D drawings of the client. Already during
these modelling activities the modeller found some errors in the clients drawings. These errors were difficult to
detect in the 2D drawings. The Project Director, engineers, work planners, and works foremen were very
enthusiastic about the 3D model. The 3D model gave a lot of insight in the power plant and its dimensions and
showed design errors. Consequently, more project participants started to use the model viewer to view the 3D model
of Pack A (and later also of Pack B). For example, works foremen used the viewer to gain better insight in the
project, to visualise the construction sequence by adding and removing construction elements, and to extract
quantities. In addition, they used the models in their meetings with subcontractors. After some weeks the
experienced drafter/modeller added cranes to the 3D model because project participants wanted to assess if cranes
could reach parts of the construction site.
379
9
th

In February, both modellers started with modelling of the cooling water intake (Pack B). The unexperienced
drafter/modeller just followed a Revit Structure training course. When the unexperienced drafter/modeller started
using Revit Structure the experienced drafter/modeller helped him to model and make form drawings. As a result,
the learning curve for the unexperienced drafter/modeller was steep. However, first the unexperienced
drafter/modeller had to spend more time on drafting in Revit Structure than he used to spend in traditional AutoCad.
According to the unexperienced drafter/modeller he would have shifted to traditional AutoCad drawings if the
experienced drafter/modeller had not supported him. Time pressure was high at that moment. The unexperienced
drafter/modeller was critical towards 3D/4D modelling first. However, after a while when he started to learn to use
Revit Structure he also started to see more benefits. He could work more efficiently in Revit Structure than in
AutoCad. He said: I dont want to use AutoCad anymore in my projects. In addition, because the
drafters/modellers made form drawings in the 3D application forms in the 3D model were kept up-to-date. This was
an important benefit compared to the models of Pack A. The 3D model of Pack A needed to be updated every time
the client sent new drawings.
At the end of February, the contractor started contract negotiations for the steelwork of Pack A with a subcontractor.
This firm appeared to be able to deliver 3D models of steelwork to the contractor. The Project Director supported
the Project Leader of this part of the project to explore the benefits and opportunities of exchanging 3D models with
the subcontractor. The Project Leader and the BIM Program Manager started up the adoption stage for this new 3D
aspect and followed the steps in Figure 3 again. Because of space limitations we are unable to describe this adoption
stage. Here we only want to mention that both the contractor and the subcontractor saw important benefits in
exchanging 3D models between their firms. Together they worked out and tested new 3D working practices. In
addition, the BIM Program Manager analysed the implementability of these new working practices and wrote an
implementation plan. At the end of June he finished the plan and sent it to the Project Director for approval. Note
that another subcontractor (for cladding) was not willing and able to deliver a 3D model. The project management
did not want to mandate the delivery of 3D models towards this firm because this would reduce competition and
raise prices.
In March, the Design Manager decided to make detail drawings of the cooling water intake in the 3D application as
well. In his view, drafting in the 3D application went successfully. However, in April he decided not to expand the
scope of the drafting activities to the cooling water outfall. Main reason was that he expected a huge workload for
coming months. This was caused by many design changes, much detailing of the constructions, and many questions
of the project team to make pictures and show things in the 3D models. It was easier and less risky for the Design
Manager to add a traditional drafter to the drafting team than a Revit Structure drafter/modeller.
In April, the design and construction firm received new drawings of Pack A from the client. Project participants
questioned if the 3D model needed to be updated now. The drafters/modellers were very busy with making drawings
of the cooling water intake. In additions, they could not make a 4D model because the project schedule was not yet
available for Pack A and B. The project management decided not to update the 3D model of Pack A and not to make
a 4D model of Pack A. There was no time available for this update and the added value of the 4D model was limited
at this moment in the project.
It is hard to determine exactly when the assimilation stage started. Theoretically this stage started if 3D/4D
technology was accepted and employed. In the assimilation stage, 3D/4D technology becomes routinised in the
activities of participants and 3D/4D technology is used to its fullest potential. However, the start of this stage differs
between actors and 3D/4D aspects. For example, with form drawings the experienced drafter/modeller moved into
the assimilation stage almost immediately after he started to use Revit Structure. However, with detail drawings this
was completely different. In addition, for the unexperienced drafter/modeller all aspects of 3D/4D modelling were
new. For him, it took some time before he accepted the use of Revit Structure for making form drawings. It even
took some additional months before he could efficiently make form drawings and this became a normal activity. At
the end of June, only making detail drawings in the 3D application were not yet routine for the drafters/modellers. In
addition, they still had to start with 4D modelling.
380
9
th

The steps taken in the implementation and assimilation stages are shown in Figure 4. First, the BIM Program Manager
executed the interventions as formulated in the implementation plan and others started to use the applications. Second,
the BIM Program Manager, Project Director, and Design Manager evaluated the interventions and use of 3D/4D
technology. Third, after a while the scope had been partly increased to other aspects (i.e., increase to detail drawings, no
increase to the cooling water outfall). This BIM Program Manager already described possible scope increases in the
implementation plan. However, for another change (i.e., the exchange of 3D models between firms) the adoption stage
had to be started up (see Figure 3). Finally, management support was needed for each change in scope or implementation
plan. These steps are in line with the theoretical implementation and assimilation stages as discussed in Section 2. Only
the importance of management support for changes in scope or plans was not mentioned in Section 2. This support was
essential for the project management to control the introduction process.
FIG. 4: Steps of the implementation and comprehension stages.
5. CONCLUSIONS
The objective of this research was to develop guidelines for the successful introduction of BIM across organisations
in a construction project. We approached the introduction and use of BIM from an IT innovation perspective.
Because the insights into these guidelines were limited, an explorative case study approach was conducted in which
we draw on the principles of action research. We developed, validated, and refined IT innovations stages and steps.
The stages and steps increased our understanding of the introduction and use of BIM in a construction project.
This study can be seen as a first step towards developing guidelines for introducing BIM among organisations in
construction projects. The guidelines (i.e., stages and steps) should be further tested in other cases. It is suggested
that subsequent research should examine projects in which (a) the use of BIM is mandated in contracts, (b) other
procurement methods are used (e.g., partnering), and (c) other types of IT application are used (e.g., workflow and
document management systems). In addition, future research should be directed towards implementation strategies. In
our case, an incremental implementation strategy was applied successfully to limit the risk of the introduction of BIM.
However, other strategies and their pros and cons should be examined as well to assess their usability in construction
projects.
The results of the study have relevance for practice as well. The stages and steps provide guidance to people
responsible for implementing BIM in construction projects on the question whether, when, and how to innovate with
BIM in their projects (Swanson and Ramiller, 2004). Increased understanding of the IT innovation process may help
these people avoiding implementation failures.
6. REFERENCES
Adriaanse, A. M. (2007). The use of interorganisational ICT in construction projects: a critical perspective.
Enschede, University of Twente. Ph.D. thesis.
Akinci, B., M. Fischer, et al. (2002). "Formalization and Automation of Time-Space Conflict Analysis." Journal of
Computing in Civil Engineering, 16(2), 124-134.
Baskerville, R. L. (1999). "Investigating information systems with action research." Communications of the AIS,
2(19), online.
381
9
th

Baskerville, R. L. and A. T. Wood-Harper (1996). "A critical perspective on action research as a method for
information systems research." Journal of Information Technology, 11(3), 235-246.
Bouchlaghem, D., H. Shang, et al. (2005). "Visualisation in architecture, engineering and construction (AEC)."
Automation in Construction, 14(3), 287-295.
Cooper, R. B. and R. W. Zmud (1990). "Information technology implementation research: a technological diffusion
approach." Management Science, 36(2), 123-139.
Harty, C. (2005). "Innovation in construction: a sociology of technology approach." Building Research &
Information, 33(6), 512-522.
Klein, K. J. and J. S. Sorra (1996). "The challenge of innovation implementation." Academy of management review,
21(4), 1055-1080.
McKinney, K. and M. Fischer (1998). "Generating, evaluating and visualizing construction schedules with CAD
tools." Automation in Construction, 7(6), 433-447.
Purvis, R. L., V. Sambamurthy, et al. (2001). "The assimilation of knowledge platforms in organizations: an
empirical investigation." Organization Science, 12(2), 117-135.
Rogers, E. M. (2003). Diffusion of Innovations, New York: Free Press.
Swanson, E. B. (1994). "Information systems innovation among organizations." Management Science, 40(9), 1069-
1092.
Swanson, E. B. and N. C. Ramiller (2004). "Innovating mindfully with information technology." MIS Quarterly,
28(4), 553-583.
Whyte, J., N. Bouchlaghem, et al. (2000). "From CAD to virtual reality: modelling approaches, data exchange and
interactive 3D building design tools." Automation in Construction, 10(1), 43-55.

382
9
th

CREATION OF A BUILDING INFORMATION MODELING COURSE
FOR COMMERCIAL CONSTRUCTION AT PURDUE UNIVERSITY

Shanna Schmelter, Office Engineer of Building Information Modeling
Holder Construction Company
& MS Graduate Alumni of Purdue University
3333 Riverwood Parkway, Suite 400
Atlanta Georgia 30339
sschmelter@hotmail.com http://www.holderconstruction.com

Clark Cory, Associate Professor
Purdue University
Department of Computer Graphics Technology
401 North Grant Street, Room 325
West Lafayette, IN 47907-2021
ccory@purdue.edu http://www.tech.purdue.edu/cgt
ABSTRACT: The focus of this study and paper was developed to determine the need for developing a course at
Purdue University with content relevant to industry trends in: Building Information Modelling (BIM) and
commercial construction practices. The outcome of the study was to develop a course at Purdue University
within the Computer Graphics Technology department that will focus on utilizing BIM for commercial
construction. Students examine 3D geometry, spatial relationships, geographic information, quantities of
materials, and properties of building components in this course. Also, this course will be designed and delivered
to assist students in learning the creation, integration, and utilization of BIM using a computer generated three-
dimensional architectural model within a commercial construction environment. The three-dimensional model,
produced by parametric software is utilized through numerous applications within a real world construction
company. The students acquire these skills by learning how BIM is used in the industry. The students also learn
the processes that make up BIM so they will be able to apply this information in a company where they can use
these techniques or change a companys ideas to incorporate BIM. It should be due to commercial trends, it is
important that students understand the concepts for future jobs. The purpose of this study was to assess the need
for and receive feedback on the syllabus of the first commercial construction computer graphics course at
Purdue University through a survey that was sent to industry professionals. With the study complete, the course
was implemented and taught during the Spring semester 2009. The paper will review the original study, talk
about the curriculum setup for the course, review the student projects and end with student comments about
course content preparing them for the AEC industry.
KEYWORDS: BIM, Construction Graphic Pedagogy, Visualization,
1. INTRODUCTION
The popularity of Building Information Modelling (BIM) in the commercial construction industry is increasing
everyday (Sullivan, 2007). A recent survey of construction projects and program owners stated that more than
one third of them used BIM on one or more of their projects. This further illustrated that educational settings are
in need of creating new courses and challenging existing ones to facilitate the need of industry (Building Design
and Construction, 2007). The educational and industrial programs that focus on construction graphics are at the
front of this need. Companies are recruiting students with computer graphics skills to BIM positions because of
the modelling knowledge. Most construction companies are slowly redefining their efforts to incorporate BIM
technology and methods. The contractors are using 3D technology to identify interferences, link data to
schedules, and produce 4D (four dimensional) animations, which help discover dynamic interferences of
construction activities (Constructech, 2007, p.25). These companies are looking for individuals straight out of
college that are knowledgeable about computer graphics and have a good sense of visualization in construction.
Merriam-Webster (2007) described visualization as the formation of mental visual images or the process of
interpreting in visual terms or of putting into visible form ( 2). It is suggested that visualization tools may help
enhance the students understanding of certain construction aspects (Messner & Horman, 2003).
383
9
th

Computer Graphic Technology (CGT) students at Purdue University who would like to specialize in
Construction Graphic Communication (CGC) are taught residential construction processes with 2D drawings
and 3D tools; however, there many more opportunities in commercial construction that are not explored by the
CGT department that could be explored in similar manners. The New Jersey Institute of Technology (NJIT)
offers a BIM class and design studio where they use Revit Architecture as the main BIM tool to further teach
BIM concepts. They are also researching how to incorporate other BIM tools such as Revit MEP and Revit
Structure into their curriculums (Autodesk, 2007).
Associate Professor of Construction Management, Willem Kymmell at Chico State University in Chico, CA has
created a BIM curriculum that has been incorporated into their school to enable students to understand BIM
concepts and become familiar with BIM tools. BIM assists in information transfer and collaboration settings.
Through this curriculum students are given the opportunity to connect how visualization, communication, and
collaboration all apply to the scope of BIM. Furthermore, the curriculum is unique because it was designed to
suit the attending audience and has basic modules developed to break BIM into manageable concepts. The
students are able to interact with industry projects where they are learning how to document construction
processes and techniques by working with industry professionals. Then the students can compare physical
observations with their BIM observations. The BIM models are still able to be manipulated after this interaction
(Kymmell, 2006).
Currently, there are no commercial construction classes taught at Purdue University through the CGT
department which if taught or introduced could be useful to these students in their future careers. The course
could potentially do several of the following: increase their visualization of 2D into 3D information, increase
their understanding of the coordination of commercial construction documentation and technology, increase
familiarities with the current technologies in BIM, increase understanding of cooperation with architects and
clients, etc. Furthermore, the lack of these visualization tools and this course may be a contributing factor to the
lack of knowledge in commercial construction processes. This course was proposed to greatly enhance the
students commercial construction knowledge through the use of BIM techniques to enhance future success in
industry.
This study evaluated the CGT 460: Building Information Modelling for Commercial Construction course
content to provide feedback from the industry who will potentially be hiring Purdue University CGT graduates.
This study collected and analyzed demographic feedback, course feedback, and general course comments to
improve the course and align it with industry standards.

2. WHO NEEDS BIM
BIM is being used in many small companies, industry leading companies, and educational arenas and they are
researching and leading new ways to establish more efficient processes and products. Small to large firms are
now incorporating BIM into their daily activities. Any size firms can have fears of disruptions, cost, and delays
from implementing BIM concepts and software. Small firms have found that transitioning to BIM needs
planning for software and workflow transition, commitment by employees to execute the plan, and measurement
of benefits or losses by using BIM. After implementing the techniques and software, there must be a continual
evaluation of projects using BIM to evaluate the benefits, what types of projects to use it on, and the scope to
use BIM on a project. Using BIM can potentially help smaller firms compete with larger firms. Small firms have
found that even though there are many culture changes and technology changes with BIM, they are still more
productive with BIM than without (Kirby, 2007).
Larger projects definitely see the benefits of BIM in multiple disciplines. At the Denver Art Museum, the
Frederic C. Hamilton Building incorporated BIM in the design and coordination which resulted in a safe and
healthy job site. The 3D model and information was used much more than the 2D documents. One reason is
because of the increased availability of BIM tools and technology to larger firms. The Denver Museum model
was originally intended for a specific piece of the structure but then used on many others (Sullivan, 2007).
Educational environments are also using BIM to teach valuable skills to be used in the construction industry.
Northumbria University of the Built Environment in the Newcastle, UK started to offer Architectural
Technology in 1998 to allow students to understand and learn how to correct the gap between design and
construction. Then in 2004 the Architectural Technology program expanded to include BIM practices. The
384
9
th

program has affected students positively due to the program interface. It is set up as the students think when
they design a structure which should include the overall scope and 3D. The lecturers found that students could
start to use Revit much quicker than traditional software. Furthermore, this trend allowed the students to learn
more about design principles and sustainability designs than the software (Autodesk, 2007c).
With BIM, fundamental information needed for the coordination of a projects design, construction and
operation is captured in digital models at the time design objects are created. The advantages that are offered by
BIM to the building industry provide strong premises to overcome the fragmented nature of the industry. As a
result the industry is likely to see new emerging processes that replace the traditional separation of design,
construction and facilities management (Salazar et al., 2006, p. 1).

3. SIGNIFICANCE OF THE STUDY
Enhanced visualization is needed to understand many complex structural components that are taught in
commercial construction. There should be a CGT curriculum that teaches commercial construction concepts and
current technology theories such as BIM to enhance students visualization of complex commercial structures.
These structures can be difficult to understand and visualize which BIM could assist. Students that are enrolled
in CGT courses that would like to have a future in commercial construction are experiencing frustration due to
the lack of formal commercial construction education. A course addressing this need using BIM would enable
students to have hands-on training with 3D tools where they can use them to further their knowledge in
operations, materials, geometry, spatial relationships, geographic information, quantities, and properties of
commercial building components. These foundations experienced in this new course may benefit CGT students
in their future career development.
The significance of this project was that CGT students will be offered the opportunity to become knowledgeable
in commercial construction and BIM which is becoming an industry standard. This educational opportunity
could potentially improve career possibilities. Similarly, commercial construction aspects of steel and concrete
are currently being taught at the Georgia Institute of Technology using BIM and parametric modelling. Charles
Eastman, a Professor in the Colleges of Architecture and Computing at the Georgia Institute of Technology,
addressed in his research the need for assessment of a new generation of architectural and building industry
design tools, especially based on parametric design and rule-based design (BIM Resources @ Georgia Tech,
2007, 20). As the construction industry and technology progress, education must be at the forefront of these
industry standards to produce the best education possible. This study assumed that by receiving and adopting
feedback from industry professionals, the course would better meet an essential part of their education.

4. PURPOSE

The purpose of the study was to demonstrate BIM techniques to educate students on commercial construction to
assist them in learning geometry, spatial relationships, geographic information, quantities and properties of
building components (Purdue University, 2007, p. 1). Students will be able to receive education on professional
practices and explain how BIM is used in the industry and the processes that make up BIM as they apply to
information in a company that uses BIM techniques (Purdue University, p. 1). There are no commercial
construction courses in CGT which ultimately can hinder a students visualization, understanding of commercial
construction concepts, and future opportunities. This proposed course may increase students knowledge,
visualization skills, and perhaps enhance their learning environment. The course objectives are (Purdue
University):

Increase spatial visualization.
Increase individual as well as group productivity.
Coordinate construction documentation and technology.
Become familiar with the current technologies in BIM and what is utilized for MEP (mechanical,
electrical, and plumbing) systems.
Understand principles of BIM and incorporate them into current projects in the AEC (architecture,
engineering and construction) industry.
385
9
th

Develop independent and teamwork skills of BIM, and know how to cooperate with architects,
clients, and everyone dealing with the need for information visualization in construction graphics.
Develop the ability to evaluate and incorporate multiple file formats into one that will evaluate
collision of MEP (mechanical, electrical, and plumbing) and structural systems within a structure.
Embed and link vital information such as vendors for specific materials, location of details, and
quantities required for estimation and tendering.
Explain how BIM is used in the industry and the processes that make up BIM as they apply to
information in a company that uses BIM techniques.
Assess a BIM project and develop a BIM project assessment.
Become aware of the career opportunities in BIM.

An evaluation tool was utilized to assess the course and laboratory material by professionals in the industry.
This study hypothesized that evaluations from the industry would prove that the course could be effective in
teaching commercial construction and BIM. Also, feedback from the professional sources assisted in adjusting
the course curriculum to be most effective for the CGT students.

5. IMPLIMENTATION OF STUDY INTO CLASSROOM
The normal classroom setup for a Construction Graphic Class at Purdue is two 1 hour lectures in which one is
based on theory and one is a demonstration of technology. The students also have a 2 hour lab toward the end of
the week. This being a 400 level course ended up being tricky to create that paralleled what is done in industry
per professional recommendations. Facility management at Purdue University was called first to obtain a set of
construction documents for one building on campus. After several weeks of negotiation on use of documents
and then scanning of original documents to obtain PDFs, the student received a complete set of construction
documents for two buildings; Knoy Hall and Electrical Engineering buildings were selected. The construction
documents did not include the MEP or Structural set of prints. Those two items were to come at a later date
which Facility Management did not identify as to a specific date.
The beginning of the semester had started and the students were put into groups of 4 in order to start the model
with the architectural PDF prints. Then the groups were paired up with 2 other groups. The idea was for one
group to work on the architectural prints using Revit Architecture- the second group was to work on the MEP
utilizing Revit MEP and the third group was to work on the structural models utilizing Revit Structural software.
Three weeks into each groups assignment, they were to switch technologies and models. So if they started
working on the architectural model, they would get the model from MEP or structural and continue to work on
it. This placed some very interesting dilemmas for each group. They had to develop a system of file naming
conventions along with create a standard of software standards in order to pick up where the other group left off.
Software standards and file naming conventions were most important items that the industrial professionals
identified as being extremely important during a project. The industrial professionals get files from multiple
subcontractors and inevitably have to utilize their models or redraw the entire structure from scratch. Either
way, software standards were to become the nightmare for each group. The file naming conventions were
eventually to become a nightmare as well. The instructor of the course made it extremely clear to each group
that the suggestions of the industrial professionals were to be taken very seriously and adhered too.
Each building is a 3 story with multiple basement levels. Each level per building was approximately 25,000
square feet, so it was imperative that the student work together to get each floor completed by the end of the 16
week semester. After each model was completed, the groups were to take them into Navisworks to run a
collision detection. Most industrial professionals are currently utilizing Navisworks in current practices;
therefore it was crucial for students to get introduced to the software. This brought another level of stress into
each students life due to the fact that they had to create 3 different models of one building and then run and
then import those modelling into a new software and try to determine how the new software runs and get a
functional report.
The overall schedule for the students to complete the class was 4 weeks for the part one of modelling. Four
weeks for the second part of modelling and 4 weeks for the third part of modelling. The last 3 weeks were to be
for Navisworks component and then a formal presentation in the 16
th
week identifying modelling through the
semester as well as a Navisworks presentation and lastly a complete summary of group functionality.
386
9
th

5.1 Creating the Architectural Model
The student were assigned to a group of 4 and then partnered with 2 other groups for a total of 12 people total.
The first job was to get the Architectural model complete. This was mainly due to the inability of facility
management to get the original structural and MEP construction documents scanned for use in the course. The
groups spent about 4 weeks on the project before the first sign of the other two sets of construction documents
were sent over. During that time, the students were able to get into work sharing and using a central file as
industry does. This allowed all 4 students to work on the same file during the same lab period. This caused a
little bit of a problem in the beginning getting used to all aspects of requesting permission to use certain parts of
the file to work on but once they overcame that lack of knowledge, progress on the architectural model started
moving very quickly. Figure 1 shows one of the architectural models created by the students in the course and
the level of detail produced. The level of detail for the interior of the Knoy Building can be seen in figure 2.
Since the students had extra time to work on the architectural model while the structural and MEP CDs were
being scanned, some spent the time to include all the exterior limestone detail while others spent time on the
inside of the structure. With the architectural model almost complete the structural and MEP scans finally
arrived. The middle of the semester was just around the corner and the students had a tremendous amount of
work to still accomplish. Both the structural and MEP documents came at the same time. This allowed the class
to work on the project exactly how the industry would. One group would work on the MEP while the other
worked on the structural model.

FIG. 1: Student Work of EE Building (Exterior Detail)

387
9
th

5.2 Creating the MEP & Structural Model
The groups were finally introduced to the MEP and the structural construction documents. This is a construction
graphics focus within computer grpahics- so this was the first time most had been able to look at a set of
construction documents specifically for MEP and structural components. Needless to say, there were several
lectures during the week to get them up to speed on how to read them. Each team accomplished the entire
structural model as shown in fig. 3. The MEP was a different story though. While most had been introduced to
MEP in previous courses, none had even drawn or modelled any of the components in the technology. It was an
eye-opening experience for all. The lectures planned were taken up by a Q&A period of MEP review. The
instructor for the course ended up modelling several components for the HVAC as well as the Plumbing in order
for them to see how the components went together in the field. And while the structural model went very
smoothly in class, the students felt the stress to get the MEP portion complete. Figure 4 shows the extent of the
MEP completed for the course. A couple of groups were able to complete the HVAC, electrical components
and Plumbing for one or 2 floors, but was not able to complete the entire building MEP.

FIG. 2: Student Work of Knoy Building (Interior Detail)

FIG. 3: Structural Model of Knoy Building

388
9
th

5.3 Collision Detection
Each group had 3 independent models in 3 different software products. The goal was to integrate each into
Navisworks and run a collision detection report. The software standards they had developed in the beginning of
the semester were put to the test. While most groups did a good job defining where the model 0,0,0 coordinate
FIG. 4: MEP Model of Knoy Building

FIG. 5: Navisworks Report (Collision Detection)

389
9
th

system was located in each model, there were a couple groups that had to go back and move everything
accordingly. Running the interference report also was a challenge in which most spent several hours in lab
during the evening just to get introduced and have an understanding of the new technology used in industry.
Figure 5 shows the typical collision detection technology report where a duct interfered with a structural
component. Each group soon found out there was more to creating BIM models than just the creation portion.
The big picture of how the model was utilized was finally sinking in and more importantly how their roles in
industry play a big part of the construction process. And while they were only able to run a couple of
comparison reports in Navisworks for a single floor, each recognized the time commitment it would take to do a
complete interference report for the entire structure.

5.4 Final Presentation
While all groups had created the entire for the Architectural, Structural and most of the MEP, they still had to
wrap it all together with a formal presentation. The presentation was to be handled as the AEC industry does
when trying to win the bid or contract for an anticipated client. The Instructor and teaching assistant were the
clients for the course and their presentation had to persuade each to sign the winning contract to the group that
presented the best overall project. Each group was given 30 minutes to introduce, present and wrap up their
project. Most groups went all out and each dressed in similar attire while others decided to take the traditional
college apparel approach. The grade of the presentation was approximately 30% of the overall grade for the
entire course, so it was taken seriously enough with the actual content. A complete review of each group
member and well as each model created was required. - along with the accomplishments & pitfalls of each over
the 15 week period. Since the course was focused on the actual BIM model, none of the groups did a set of
formal construction documents from the model, but focused on each model specifically. Renderings were also
created as a means to present what was accomplished during the semester as shown in figure 6 of an interior and
exterior.

FIG. 6: Exterior & Interior Rendering

390
9
th

6. CONCLUSIONS
The overall course defined and created from industrial recommendations of BIM modeling was an overall
success. While there were glitches in the beginning due to lack of information, the overall course went very
smoothly. According to end of year evaluations, each student thought the class was challenging but very helpful
in their education and would benefit them later while applying for jobs in the AEC industry. The overall
schedule for the class was changed multiple times through the semester due to lack of information obtained or
lack of knowledge on students understanding of MEP or structural print reading but in the end each came away
with a greater understanding of their future roles in the AEC industry.
The course will be modified slightly to include a complete set of construction documents presented at the end of
the course. Since it is a construction graphics curriculum, it makes common sense to include a set of working
construction documents to be submitted since each student will be producing those every day in industry. Also,
there will be smaller projects. The 2 buildings selected were extremely large in size to accomplish everything in
one semester. Future building projects will be the size of the building selected for this course. The group sizes
will also be cut in half. This will force each student to take on more responsibility as well as get introduced to
more technology.
Each modification was a direct result of information obtained by industry, the instructors observation and
experience in AEC industry, or student comments from semester evaluations. Surprisingly, most students were
very vocal when it came to course input for future course content. While most are wrapped up with finishing
finals or interviewing for jobs, these students were trying to help create a course that would benefit future
students. The instructor was pleased to read so many comments about course improvements. The students that
took the class for the first time all mentioned that they see the benefits industrial input and the comments they
included were a way to give back to Purdue for all it has done for them.
The course created from industrial experience and input is an ongoing process at Purdue University in the
Construction Graphics area of focus. Real world experiences with hands on focus are how the technology
students learn best. The AEC industry is going through the greatest changes it has seen in the last 25 years, the
curriculum at Purdue will remain in constant state of change and update due to technological advancements and
how the BIM is utilized in the AEC industry. With the help of industry, the students are receiving the best
education possible.

7. REFERENCES
Autodesk. (2007). Revit building information modelling: BIM goes to school [white paper, electronic version]
http://students2.autodesk.com/ama/orig/BIM_Goes_To_School.pdf
BIM Resources @ Georgia Tech. (2007). http://dcom.arch.gatech.edu/chuck
Building Design and Construction. (2007). BIM adoption accelerating, owners study finds.
http://www.bdcnetwork.com/article/CA6500734.html?nid=2073
Constructech. (2007). BIM builds its case, Constructech Magazine, Vol. 10 No. 9, 25-28.
Kirby L. (2007). Introducing BIM into a small-firm work environment, Small Project Practitioners Knowledge
Community Journal: The American Institute of Architects, 42.
Kymmell W. (2006). Outline for a BIM curriculum.
http://www7.nationalacademies.org/FFC/willem_kymmell_csu.pdf
Messner J. and Horman M. (2003). Using advanced visualization tools to improve construction education,
Proceedings of the CONVR 2003, Conference on Construction Applications of Virtual Reality.
Purdue University. (2007). CGT 460: Building information modelling for commercial construction ,
http://www2.tech.purdue.edu/cgt/courses/cg460
Salazar G., Mokbel H., Aboulezz M. and Kearney W. (2006). The use of the Building Information Model in
construction logistics and progress tracking in the Worcester Trial Courthouse, Proceedings of the Joint
391
9
th

International Conference on Computing and Decision Making in Civil and Building Engineering,
Montreal, Canada, 986 995.
Sullivan C. (2007). Integrated BIM and design review for safer, better buildings: How project teams using
collaborative design reduce risk, creating better health and safety in projects. McGraw Hill Construction
Continuing Education. http://construction.com/CE/articles/0706navis-3.asp

392

CONVR 2009 Proceedings

Încărcat de

Informații document

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

CONVR 2009 Proceedings

Încărcat de

Drepturi de autor:

Formate disponibile

CONVR 2009

III. SIMULATION AND ANALYSIS

Changing Collaboration in Complex Building Projects

Bio of Dr. AbouRizk:

Bio of Prof. Dawood:

3.2.3 Measurement of design protocols and testing hypotheses

FIG. 3 Case study project Design Scenario Model (DS)

TOWARD AFFECTIVE HANDSFREE HUMAN-MACHINE INTERFACE

FIG. 4: Visual Information Exploration

FIG. 6: Time-Controlling Maneuver

FIG.7: Information Filtering

FIG. 8: Information Emphasizing

FIG. 9: Information Searching

AUTOMATIC GENERATION OF TIME LOCATION PLAN IN ROAD

2.2.2 Development Conflict Identification System (CIS):

2.2.4 Demonstration of conflict points calculation:

3.2 Time Location Plan Generated by the VCM:

VIRTUAL ENVIRONMENT ON THE APPLE IPHONE/IPOD TOUCH

3.1 The Unity game engine

4.5 Interaction techniques

S-ar putea să vă placă și