Journal of Computational Design and Engineering 6 (2019) 179–188

Contents lists available at ScienceDirect

Journal of Computational Design and Engineering

journal homepage: www.elsevier.com/locate/jcde

An indoor thermal environment design system for renovation using

augmented reality
Tomohiro Fukuda ⇑, Kazuki Yokoi, Nobuyoshi Yabuki, Ali Motamedi
Division of Sustainable Energy and Environmental Engineering, Graduate School of Engineering, Osaka University, Suita, Osaka, Japan

a r t i c l e i n f o a b s t r a c t

Article history: The renovation projects of buildings and living spaces, which aim to improve the thermal environment,
Received 4 April 2018 are gaining importance because of energy saving effects and occupants’ health considerations. However,
Accepted 29 May 2018 the indoor thermal design is not usually performed in a very efficient manner by stakeholders, due to the
Available online 2 June 2018
limitations of a sequential waterfall design process model, and due to the difficulty in comprehending the
CFD simulation results for stakeholders. On the other hand, indoor greenery has been introduced to build-
Keywords: ings as a method for adjusting the thermal condition. Creating a VR environment, which can realistically
Environmental design
and intuitively visualize a thermal simulation model is very time consuming and the resulting VR envi-
Indoor thermal environment
Intuitive visualization
ronment created by 3D computer graphics objects is disconnected from the reality and does not allow
Interactive environment design stakeholders to experience the feelings of the real world. Therefore, the objective of this research
Computational Fluid Dynamics (CFD) is to develop a new AR-based methodology for intuitively visualizing indoor thermal environment for
Augmented Reality (AR) building renovation projects. In our proposed system, easy-to-comprehend visualization of CFD results
augment the real scenes to provide users with information about thermal effects of their renovation
design alternatives interactively. Case studies to assess the effect of indoor greenery alternatives on
the thermal environment are performed. In conclusion, integrating CFD and AR provides users with a
more natural feeling of the future thermal environment. The proposed method was evaluated feasible
and effective.
Ó 2018 Society for Computational Design and Engineering. Publishing Services by Elsevier. This is an open
access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

1. Introduction
The renovation projects of buildings and living spaces are
attracting attention in the architecture, engineering and construc-
tion industry (AEC) as they replace construction project for new
buildings and reduce the cost, decrease environmental impacts,
and maintain neighborhood relationships (Pombo, Rivela, & Neila,
2016; Kovacic, Summer, & Achammer, 2015). Recently, improving
the thermal environment performance bring heightened attention
at renovation projects. Saving building energy is mean against glo-
bal warming (Voss, 2000; Holopainen, Milandru, Ahvenniemi, &
Hakkinen, 2016). Considerations for occupants’ health are
other reason for performing renovation projects to decrease
accidents by indoor heat stroke and heat shock (Laschewski and
Peer review under responsibility of Society for Computational Design and
Engineering.
⇑ Corresponding author.
E-mail addresses: fukuda@see.eng.osaka-u.ac.jp (T. Fukuda), u759379c@alumni.
osaka-u.ac.jp (K. Yokoi), yabuki@see.eng.osaka-u.ac.jp (N. Yabuki), ali.motamedi@
see.eng.osaka-u.ac.jp (A. Motamedi).
Jendritzky, 2002; Thomsen et al., 2016; Zhu, Ouyang, Cao, Zhou,
& Yu, 2015). To tackle energy saving issue, decision support
systems for energy performance improvement of the sustainable
building renovation have been proposed (Juan, Gao, & Wang,
2010; Nielsen, Jensen, Larsen, & Nissen, 2016). To tackle occupants’
health issues, a PMV (Predicted Mean Vote) model was proposed as
a comfort indicator in an indoor environment (Fanger, 1970;
Fanger and Toftum, 2002). However, the indoor thermal environ-
ment for new or renovated projects are not still satisfactory for
some owners and research efforts on the human comfort in the
indoor environment continues (Rupp, Vasquez, & Lamberts,
2015). This is partially because the indoor thermal design is not
usually performed in an efficient manner by the owners, architects,
facility designers and engineers.
One of the issues is that the owner can not sufficiently partici-
pate in the discussion in the design process of the indoor thermal
environment. One reason is that the building design usually fol-
lows a sequential waterfall model. Because the thermal environ-
ment design process is one the latest stages, major changes to
the design for the thermal environment design improvement are
difficult and costly to apply. Recently, the concept of integrated

https://doi.org/10.1016/j.jcde.2018.05.007
2288-4300/Ó 2018 Society for Computational Design and Engineering. Publishing Services by Elsevier.
This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
180 T. Fukuda et al. / Journal of Computational Design and Engineering 6 (2019) 179–188
project delivery (IPD) was promoted as a new design process and a
new alternative to the waterfall model (Ghassemi and Gerber,
2011). IPD optimizes the building design process, wherein stake-
holders such as owners or designers cooperate with each other
from the beginning to the end of the design project. Building infor-
mation modeling (BIM) is essential for realizing IPD (Eastman,
Teicholz, Sacks, & Liston, 2011). BIM is a standard information
model that can be used for various simulations related to buildings
in the early design stage (Lee and Song, 2010).
Another reason for the inefficiency in collaboration is that the
visualization results generated by CFD (Computational Fluid
Dynamics) software applications are not easy to interpret by non-
professionals of thermal environment design. CFD simulation is
used to predict and assess building’s thermal environment in
the design phase (Den Hartog, Koutamanis, & Luscuere, 2000).
The CFD results usually comprise a large amount of numerical
data, and they are difficult to comprehend in relation with the
three-dimensional (3D) space. This affects the efficiency of discus-
sions related to the thermal design. The results are generally visu-
alized in three dimensions by analysis software applications.
These applications can analyze the CFD results in detail; however,
they cannot realistically visualize the simulation model. Addition-
ally, scenes can only be visualized in a third-person view in such
applications. Therefore, the visualization of the results in the vir-
tual reality (VR) environment using a game engine has been pro-
moted as a method for solving above-mentioned problems. A
game engine is a software framework designed for creating and
developing video games and it includes a rendering engine for
3D graphics, a physics engine, and localization support. VR using
a game engine can provide photorealistic 3D graphics in a first-
person view. Hence, CFD visualization in VR using a game engine
can be efficiently utilized for studying both thermal and architec-
tural environment designs (Hosokawa, Fukuda, Yabuki,
Michikawa, & Motamedi, 2016). However, making a numerical
3D building model including existing buildings components and
furniture to express realistically in a VR environment is expensive
and is very time consuming. Additionally, a VR environment cre-
ated by 3D computer graphics objects is disconnected from the
reality and does not allow design stakeholders to experience the
feelings of the real world. Augmented reality (AR) overlays
images, 3D virtual objects, or computer-generated data on real-
world images using video or photographic displays. Specifically,
AR can help to visualize full-scale design projects on a construc-
tion site. This technology has the advantage of utilizing real-
word scenes of existing rooms. Hence, CFD visualization using
AR technology can be efficiently utilized for examining the out-
comes of renovation projects.
Indoor greenery has been used in buildings as a renovation
method to improve thermal environment (Bregon, Urrestarazu, &
Valera, 2012). Indoor greening is known by activities such as
installing plants on walls and/or ceilings, hanging plants, and
installing potted plants in the room. It is often used for providing
a stress-relief environment because of the effect of the green color.
Meanwhile, plants are known to improve the indoor thermal envi-
ronment as they can be used to adjust the temperature, and has
effect on the humidity level of the room due to transpiration action
of plants (Asaumi, Nishina, Kei, Masui, & Hashimoto, 1994).
The objective of this research is to develop a new AR-based
methodology for visualizing indoor thermal environment for build-
ing renovation projects. In our proposed system, easy-to-
comprehend visualizations of CFD results augment the real scenes
to provide users with information about thermal effects of their
renovation design alternatives interactively. Case studies to assess
the effect of indoor greenery on the thermal environment are
performed.
2. Previous studies
CFD enables the analysis of thermal distribution and airflow for
use in architectural thermal environment design. Researchers and
practitioners can evaluate building performance from the design
stage using CFD to solve problems and to create advanced designs
(Gan, 1995; Nielsen, 2004). Since CFD results are huge numerical
data sets, transformation of the numerical data into visual repre-
sentations by post-processing is necessary. The available visualiza-
tion software applications target engineers, who are familiar with
CFD; therefore, evaluating CFD results becomes difficult for non-
professionals. Hence, displaying CFD results using models created
in BIM or 3D-CAD in a VR environment, which is a highly interac-
tive medium, has been proposed (Savioja et al., 2003; Bahar,
Jeremie, Christian, & Nicolle, 2015; Bille, Smith, Maund, &
Brewer, 2014; Matthias and Veruna, 2015). In some projects, CFD
results were visualized in the VR environment where the architec-
tural design was represented to understand the design better and
in a more sophisticated design environment. Fukuda, Mori, &
Imaizumi (2015) proposed a system that integrates commercial
CFD and VR software applications to display CFD results in a VR
environment. However, the research showed some issues with
the VR presentation of thermal simulation results. For example,
the vector data converted from CFD to VR did not retain the arrow
information representing wind-flow and the color information rep-
resenting temperature distribution.
On the other hand, design visualization using AR has also been
investigated (Webster, Feiner, MacIntyre, Massie, & Krueger, 1996;
Goldsmith Liarokapis, Malone, & Kemp, 2008; Uva, Cristiano,
Fiorentino, & Monno, 2010; Gunnarsson, Rauhala, Henrysson, &
Ynnerman, 2006; Re et al., 2014; Yabuki, Miyashita, & Fukuda,
2011). Nakagawa et al. (2015) visualized the online and offline
heat-maps of object temperatures using AR. In their study, the
temperature differences for objects were also displayed to enable
users to understand heat changes. However, the research does
not support CFD simulation. Malkawi and Srinivasan (Malkawi
and Srinivasan, 2005) introduced a human–building interaction
(HBI) model that allows exploration of CFD data sets in real time.
An immersive AR model designed to visualize and interact with
the thermal models was used. However, the initial conditions
could not be changed to output different visualizations. Yabuki,
Furubayashi, Hamada, and Fukuda (2012) realized AR visualization
of CFD analysis results in a large outdoor space.
Indoor greening has attracted attention as an environmental,
psychological and social friendly design methodology (Tan,
Wong, Tan, Ismail, & Wee, 2017; Montacchini, Tedesco, &
Rondinone, 2017; Rogerson, Gladwell, Gallagher, & Barton, 2016).
Researches on predicting outdoor thermal environment by green-
ing by using numerical analysis has been reported in the field of
architecture and urban environmental engineering. Yoshida,
Ooka, Mochida, Tominaga, and Murakami (2000) modeled a new
3D plant canopy model for urban area and calculated the thermal
environment around trees. The tree model was input to the 3D
convection, radiation and moisture transport coupled analysis of
the urban area, and investigated the influence on the thermal envi-
ronment in the summer based on the type and scale of the green
area. Srivanit and Hokao (2013) conducted CFD analysis to mea-
sure the cooling effect of trees and green areas in the university
campus. As a result, when trees were increased by 20% from the
present condition, the average temperature decreased by 2.27 °C.
In this research an environmental design system is developed
that can simultaneously visualize both the thermal environment
and the appearance of the architectural space of the proposed
renovation plan with indoor greenery. In order to enhance the
scalability of the system, an open source software is used as CFD
 T. Fukuda et al. / Journal of Computational Design and Engineering 6 (2019) 179–188
analysis and game engine is used for the development of AR mod-
ule. In addition, Head Mounted Display (HMD) is used to improve
immersion. Furthermore, an interface is designed and imple-
mented that can interactively change parameters and generate dif-
ferent results during AR runtime. In order to handle the input
operation when the HMD is attached, a game pad is used instead
of the keyboard and the mouse.
3. Indoor thermal environmental design system using AR
In our proposed system, CFD and AR are integrated. It comprises
five steps, i.e., BIM model creation, mesh generation, setting of
boundary conditions, CFD simulation, and AR visualization. First,
a BIM model of the building is created using a BIM authoring soft-
ware application. Next, the volume mesh of the BIM model is cre-
ated using mesh-generation software applications. Boundary
conditions are set based on model information, such as the heat
flux of walls and the amount of airflow from air-conditioners. In
the simulation step, non-steady-state fluid simulation is performed
using a CFD software application based on the defined boundary
conditions. The simulation results are converted and visualized
using AR technology in the visualization step. The 3DCG model of
CFD analysis results and the architectural space of the renovation
plan, and the live video of the current space are superimposed in
the AR environment. The CFD analysis results and AR visualization
results are changed in accordance with changes in renovation pro-
posals such as indoor greening. The system flow for the AR envi-
ronment visualization is shown in Fig. 1. The details of each step
are described in the following subsections.
3.1. BIM model and mesh generation
The BIM model is created using a BIM application software.
Since the BIM model is used for CFD analysis, the geometry of
detailed building components and furniture are also created. In
addition, attribute information, such as the heat transfer coefficient
of the walls and the transmittance of the window, which are nec-
essary for the CFD analysis, are input.
Based on the created model, the volume mesh is created using a
mesh generation software. First, the model is split to structured
grids. Next, the grids are split to smaller grids around the objects
such as an air-conditioner. In the case of indoor thermal environ-
ment analysis, it is not necessary to create inner meshes of building
components that are not subject to calculation. Creating detailed
meshes increases the time for the CFD analysis, yet increases the
accuracy of CFD analysis.
3.2. Boundary condition setting and CFD simulation
Boundary conditions are set based on the model information,
such as the heat flux of the wall and the amount of airflow from
Fig. 1. Data flow of the proposed system.
181
an air-conditioner. The settings for heating, ventilating, and
air-conditioning (HVAC) are manually determined, and the bound-
ary conditions of inlet and outlet of the air-conditioner and venti-
lation devices are input in the VTK (Visualization Tool Kit) format.
The airflow of the air-conditioner and ventilation devices is set
based on the box method (Nielsen, 2004). The voxel meshes that
cover the geometry of HVAC components geometry are generated,
and the thermal property of each mesh is recorded in the VTK for-
mat. A radiation model is implemented to the solar model includ-
ing the solar primary heat fluxes, the reflective fluxes on walls, and
the diffusive sky radiative fluxes. The sky diffusive radiations for
horizontal and vertical walls are calculated according to the Fair
Weather Conditions Method from the ASHRAE Handbook
(ASHRAE, 2009). The information related to the solar direction such
as Greenwich Mean Time (GMT), start date, start time, longitude,
latitude, and the radiative properties such as emissivity, absorptiv-
ity, and transmissivity of the architectural components are set
manually. Relative humidity is calculated based on the ASHRAE
Handbook (ASHRAE, 2009).
The thermal conditions of the indoor environment are simu-
lated using a CFD software application such as OpenFOAM, which
is an open-source CFD software toolbox. The thermal simulation
is performed on the assumption that the fluid is non-steady state
and has turbulence flow. The CFD solver calculates the heat trans-
mission using a coupled analysis of solid heat conduction and fluid
heat transfer and solid-fluid heat transmission. Each region of solid
and fluid are allocated from the grids created in the mesh genera-
tion step. In detail, multi-region thermal fluid analysis of fluid-
solid thermal coupling is performed for CFD analysis including
solar radiation. Multi-region thermal fluid analysis solves heat
transfer by fluid, heat transfer by solid, and heat transfer between
fluid and solid. For the coupling method of pressure and velocity,
the PIMPLE method is used in the unsteady state (Holzmann,
2017). The turbulence model uses the k-e model. Solar radiation
models includes solar heat flux, wall reflection flux, radiation diffu-
sion flux. Sun solar radiation flux is calculated using face shading
algorithm. Radiant diffusion flux is calculated based on The Fair
Weather Conditions Method (ASHRAE, 2009). Also, humidity anal-
ysis is performed in a non-compressible steady state with buoy-
ancy consideration. After executing the simulation, a post-
processing activity using a visualization software application, such
as Paraview, is performed to check the results.
3.3. Visualization using AR
Simulation results are visualized on the scenes of the real
indoor environment using the AR technology. In order to represent
thermal distribution, XY, YZ, and ZX planes of heat-map, which are
suitable for representing the 3D thermal distribution, were used.
Arrows were used to express air characteristics; the wind direction
is represented by the direction of the arrows, and wind velocity is
represented by their length. An input device such as a gamepad
182 T. Fukuda et al. / Journal of Computational Design and Engineering 6 (2019) 179–188

controller enables users to operate the visualization environment Table 1

interactively by changing the visualization parameters. Boundary conditions (building components).

To accurately render the AR, the geometric registration between Wall Ceiling Floor Window
a live video and the 3D object must be precisely tracked, which 2
Heat transmission coefficient [W/(m
K)] 1.38 1.80 1.70 5.60
remains a challenging problem. The reference point for geometric Indoor wall temperature [°C] 20.0 20.0 20.0 20.0
registration corresponds to the position of the sensor in sensor- Outdoor temperature [°C] 11.6 20.0 20.0 11.6
based methods (such as GPS or 3D sensors) (Schall et al., 2009; Emissivity 0.700 0.700 0.700 0.113
Absorptivity 0.700 0.700 0.700 0.113
Hii, Zhou, Karlekar, Schneider, & Lu, 2009; Fukuda, Zhang, Yabuki, Transmissivity 0.010 0.010 0.010 0.815
2014) or the position of an artificial marker in marker-based

Table 2
Boundary conditions (air-conditioner).

Inlet Outlet
Air-volume [m3/min] 18 18
Area [m2] 0.06  4 0.36
Velocity [m/s] 2.5 1.2
Temperature [°C] 28 –
Inflow angle [°] 60 –

methods (that is an example of vision-based methods) (Re et al.,

2014). In sensor-based methods, the reference point of geometric
registration is normally positioned near an AR user, and thus the
sensing error significantly influences the accuracy of 3D object reg-
istration. In artificial marker-based methods, it is possible to place
large markers near target objects. However, it is necessary for the
marker to be properly captured by the AR camera, and this imposes
limitations on the user’s movability range, and requires the place-
ment of large markers. Therefore, many studies continue to focus
on developing marker-less AR systems that do not require sensors
and artificial markers (Klein and Murray, 2007; Ventura and
Hollerer, 2012; Schubert, Schattel, Tonnis, Klinker, & Petzold,
2015). Yabuki, Yabuki, Hamada, and Fukuda (2012) developed a
robust geometric registration method of a marker-less outdoor
AR that corresponds point cloud data to natural features of the real
world. However, the system requires special equipment such as a
3D laser scanner to detect these point clouds. The advantage of
HMDs as AR displays is that they provide an immersive experience
compared with other types of displays such as hand-held. There
Fig. 2. BIM model (M3-411).

Fig. 3. Mesh generation (M3-411).

 T. Fukuda et al. / Journal of Computational Design and Engineering 6 (2019) 179–188 183

Fig. 4. Room M3-411: Floor plan with two transmitters of the infrared ray (left), actual photo (right).

Fig. 5. Indoor greening scenarios: (b) window, (c) ceiling, and (d) partition.
184 T. Fukuda et al. / Journal of Computational Design and Engineering 6 (2019) 179–188

Table 3 Table 5
Boundary conditions (components with greenery). Simulation results (Humidity).

Window Ceiling Partition No greenery Window Ceiling Partition

2
Greenery area [m ] 30.77 143.86
Heat transmission coefficient [W/(m2
K)] 4.43 1.66
Emissivity 0.101 0.408
Absorptivity 0.101 0.408
Transmissivity 0.089 0.009
are two types of HMDs, i.e., video see through type and optical see-
through type.
In this research, since AR is used in an indoor environment, a
marker-based registration method with ease of use and high regis-
tration accuracy in 1/1 scale is used. Additionally, for the use of the
AR system in a large-scale indoor environment, a sensor-based reg-
istration method that allows free movement of the viewpoint is
used.
3.4. Design feedback
The design process using the proposed system enables the com-
parison of CFD results. The user analyzes the CFD results that are
visualized in the AR environment along with the building features
and discusses changing the configurations of the building and
HVAC components. The user can change the parameters, and the
system performs new simulations and visualizations accordingly.
The user can compare scenarios by switching between visualiza-
tion outputs. The user can change the setting by either selecting
from predefined values provided by the system or by changing
CFD parameters. Predefined setup results are pre-calculated by
the system and are readily available to the user. In contrast,
changing individual parameters requires processing time for
visualization.
Consequently, by repeatedly changing settings, various AR visu-
alizations of indoor thermal environment can be achieved, ana-
lyzed, and compared by stakeholders in a meeting. This aids
identifying better design alternatives, which leads to the improve-
ment of building performance.
4. Implementation
A prototype system was developed for validating the methodol-
ogy proposed in this research. The verification experimentation
was performed in a room 411 (L-shape room: max. dimensions:
18.5  10.4  2.6 m (width  depth  ceiling height)) of M3 build-
ing at Osaka University Suita Campus (34.822°N, 135.521°E). The
experiments were performed in January 2017 during winter in
Japan.
The system flow is shown in Fig. 1. The BIM model was created
using Autodesk Revit Architecture 2016 (see Fig. 2). In Revit, CFD-
related attributes, such as a heat transfer coefficient, thermal resis-
tance, and absorption rate to calculate heat transfer can be input.
The created model is exported as an industrial foundation classes
(IFC) file, and is converted to the STL format for generating the
mesh for CFD simulation. The analysis area and the interface
between the three air conditioning areas and the window surface
are defined. In the analysis area, after defining large and small rect-
angular parallelepipeds by blockMesh software, an L-shaped room
can be defined by combining them. For the air conditioning, an
35.54 Humidity [%] 26.21 30.19 34.00 26.40
1.38
0.408
inlet with an area of 0.36 m2 was set at the center and four outlets
0.408
0.009 of 0.36 m2 were set on each side of the inlet. The mesh pitch was
uniformly generated at 13.5 cm across the entire space (see
Fig. 3). The boundary conditions of the building components,
including the heat transmission coefficient, indoor wall tempera-
ture, outdoor temperature, and heat transmission rates, were set
as presented in Table 1. The boundary conditions of the air-
conditioning unit, including the flow rate, area, velocity, turbulent
intensity, temperature, and supply angle, were set as presented in
Table 2.
The CFD simulation was executed using OpenFOAM (ver.2.2,
ver.1606+). The simulation results were exported in the VTK file
format and were imported to Unity (Ver.5.3.2.f). Before the main
experiment, the temperatures by the CFD simulation were com-
pared with the actual temperature values to evaluate the CFD per-
formance. Seven temperature sensors and three temperature and
humidity sensors are installed at ten data collection points in five
locations in the room M3-411 to measure temperature and humid-
ity. The sensors positions were arranged at almost equal intervals
in the room. However, the locations where the indoor temperature
largely varies, such as near the PCs were excluded. The actual
temperature values were measured from 12:00 to 17:00 between
January 25th and February 6th 2017. The difference between the
actual temperature measurements and the CFD simulation results
was within 1 °C at 4 for the 10 sensing points. Humidity was calcu-
lated as one value by the CFD analysis for the room. The average
value of the relative humidity for the three humidity sensors was
33.1%, and that of the CFD analysis was 26.21%.
For the AR visualization, HMD for AR mounts Ovrvision Pro as a
USB3.0 stereo live camera in Oculus Rift DK2 is used and HTC Vive
as popular HMDs for VR is used. AR camera (Ovrvision Pro) was
calibrated beforehand for distortion correction. A large AR marker
(0.72  0.72 m2 (height  width)) is used for the marker-based AR
with 1/1 visualization was set at the center of the room M3-411,
and the user wore Oculus Rift DK2 with Ovrvision Pro by standing
near the wall on the east side. The height of the marker was set to
1.75 m for better visibility. For the sensor-based AR, two infrared
sensors were installed in the center of the room, which create an
operating area of 4.3  3.8 m2 (width  length). The height of each
sensor was set to 2.1 m to detect a standing user. The sensors were
tilted vertically by about 10° to be able to capture the floor (see
Fig. 4). 3DCG models were visualized using Ovrvision Pro SDK
Unity plug-in. The system was operated using Logicool f710 game
pad controller. A desktop PC with Intel Core i5 4670 K @ 3.40 GHz
of CPU, 16.0 GB Dual-Channel DDR3 @ 798 MHz of RAM, 2047 MB
NVIDIA GeForce GTX 650 of GPU, running Windows 8.1 Enterprise
64-bit is used for the CFD analysis.
The following four indoor greening scenarios were analyzed for
the room M3-411 as renovation proposals: (a) no greenery, (b)
greenery on window side, (c) ceiling with greenery, and (d) parti-
tions with greenery. The greening areas of all renovation proposals
are shown in the Fig. 5 and the information about the greening area
and the greening boundary conditions are presented in Table 3.

Table 4
5. Results
Simulation results (Temperature).

No greenery Window Ceiling Partition 5.1. CFD analysis results

Maximum [°C] 28.00 28.00 28.00 30.71
Minimum [°C] 21.10 21.48 21.94 24.05 The simulation results of temperature for the greenery scenar-
Average [°C] 21.98 22.25 22.80 27.19
ios are shown in Table 4. Comparing the average temperature with
 T. Fukuda et al. / Journal of Computational Design and Engineering 6 (2019) 179–188
Fig. 6. AR visualization results of indoor thermal environment in room M3-411.
no greenery scenario, the average temperature increased for about
0.27 °C in the window scenario, 0.82 °C in the ceiling scenario and
5.21 °C in the partitions scenario.
The simulation results of humidity are shown in Table 5. Com-
paring the humidity with no greenery scenario, the humidity
increased for 3.98% in the window scenario, 7.79% in the ceiling sce-
nario and 0.19% in the partitions scenario.
5.2. AR visualization results
Fig. 6 shows snapshots of the AR visualization of the indoor
thermal environment of room M3-411 with Oculus Rift DK2. The
developed system recognized the AR marker, and the 3DCG model
of the thermal environment was superimposed and tracked in the
space. In the 3DCG model, the wind direction is represented by
arrow symbols and heat maps represent the temperature. The
arrow symbols, which represent the wind direction, are colored
according to the wind velocity, ranging from red for high and blue
for low velocity. As shown in Fig. 6, the heat map and the color of
185
arrows are red in the vicinity of the air conditioning which refers to
high temperature and high wind speed. The user can intuitively
understand such information using the system.
Fig. 7 shows AR visualization results in which 3DCG model of
the thermal environment and the greening were superimposed
on the real scene for indoor greening scenarios (i.e., window side
with greenery, ceiling side with greenery, partitions with green-
ery). The superimposition of the greening model made it possible
to study the indoor landscape. Additionally, in visualizing the air-
flow, the user can see that the direction of the airflow is changed
by placing the partitions with greenery in the room.
Fig. 8 shows AR visualization results of the indoor thermal envi-
ronment of room M3-411 with HTC ViVE, which is a sensor-based
registration methods.
6. Discussion
The results of the experiment showed that the proposed
approach is feasible for indoor thermal and environmental design
186 T. Fukuda et al. / Journal of Computational Design and Engineering 6 (2019) 179–188

Fig. 7. AR visualization results of indoor thermal environment in room M3-411.

analysis. A user (e.g. owner, architect, facility designer and engi-
neer) can create and study both indoor thermal environment and
the appearance of the architectural space of the renovation plan
using AR. Easy-to-comprehend visualization of CFD results aug-
ments the real scenes to provide users with information about
thermal effects of their renovation design alternatives aiming for
better environmental design.
6.1. CFD analysis
The results of CFD analysis showed that the temperature and
humidity increased in the room. The experiment presented in
(Asaumi, Nishina, Kei, Masui, & Hashimoto, 1994) reported that
both temperature and humidity increases with indoor greening,
and the CFD analysis results in this study confirmed this effect.
The reason for the temperature increase is that the greening
surface is warmed by the solar radiation, and the heat transfer to
the air has bigger effect than sunlight shielding effect of greening.
In addition, in the case of no greening, the amount of heat transfer
to the indoor air is small because the solar radiation reaches the
floor and walls directly without being blocked by greenery, and
the heat transfer coefficient of the floor and walls is small. The rea-
son for the humidity increase is that the parameter that took tran-
spiration effects into account is defined for the surfaces with the
greenery. In the case of partitions with greenery, the increase in
humidity was small. The reason is that saturated water vapor
and saturated water vapor pressure are increased as a result of
the rise in the room temperature. The temperature increase is
related to the heat quantity of the installed partitions affecting
the air inside the room.
The long calculation time was identified as a limitation for the
CFD analysis as an early design study tool. In our experiment,
calculation time for one of the scenarios for the CFD analysis was
about 3 h in CPU time and about 15 h in actual elapsed time. As
 T. Fukuda et al. / Journal of Computational Design and Engineering 6 (2019) 179–188
Fig. 8. Visualization results obtained using sensor-based AR.
an alternative, Fast Fluid Dynamics (FFD) solver was developed
based on a numerical scheme (Stam, 1999) and an indoor environ-
ment simulation using FFD was implemented (Zuo, Wetter, Tian, Li,
Jin, & Chen, 2015). Although FFD is less accurate, the calculation
speed of FFD is higher than that of CFD. Hence, new technologies
for faster CFD calculation with satisfactory accuracy to allow iter-
ative design feedback for new design alternatives should be
explored.
6.2. AR visualization
The accuracy of the marker-based AR registration was verified
by comparing the actual position of objects with the location of
the augmented 3D model. The horizontal and the vertical errors
were calculated in 10 different photographs. The results show that
the average of the horizontal errors and the vertical errors were
2.49 pixels and 3.06 pixels, respectively. These errors were small
and are caused by reasons such as the image-capturing issues of
the Ovrvision Pro camera. The camera on the HMD always slightly
moved or it was not stable because of vibrations. Additionally, the
lighting conditions and the distance between the AR camera and
the marker affected the accuracy. Moreover, the viewing area
was limited as the marker had to be constantly in the field of view.
Regarding this limitation, the sensor-based AR registration showed
more robustness. The 3DCG displayed in HMD (ViVE) was stable
with small vibration as long as the users stayed within the tracking
area. Similar to the visualizations in Figs. 6 and 7 obtained by the
marker-based method, thermal models and greenery models aug-
mented the real-world scene. The largest tracking area has a diam-
eter of 5 m. The user cannot walk beyond this range, which causes
limitations.
The proposed indoor thermal environment design system could
reduce the explanation time for the renovation design plan and
simplify the explanation of the relation between architectural
design and thermal environment. Furthermore, the proposed sys-
tem was particularly effective when designing a space for which
thermal environment prediction is important and difficult, such
as large spaces, large halls, server rooms with heat-generating
equipment, complicated structures, and rooms.
187
7. Conclusion
In this research, a new methodology to support indoor thermal
environment design by integrating CFD and AR is proposed and
implemented to facilitate developing a collaborative design envi-
ronment among stakeholders such as owners, architects, facility
designers and engineers. The BIM model was utilized together with
the CFD simulation considering the solar radiation. The results of
the CFD simulations were visualized with arrows and heat-maps
using AR technology. Case studies were performed for the renova-
tion projects with indoor greenery, and the simulation results were
compared to the case with no greenery. The thermal results and
the greenery model augmented the scenes of the actual environ-
ment when using the prototype system.
Users (e.g. owners, architects, facility designers and engineers)
can create and study both indoor thermal environment and the
appearance of the architectural space of the renovation plan using
the proposed system. Integrating CFD and AR provides users with a
more natural feeling of the future thermal environment. Easy-to-
comprehend visualizations of CFD results augment the real scenes
to provide users with information about thermal effects of their
renovation design alternatives towards better environmental
design.
As future work, more functions and detailed settings will be
considered for more accurate CFD simulation. Additionally, simula-
tion and visualization of other factors, such as air quality or sound
insulation, are required for more extensive studies. The case stud-
ies should be performed in larger spaces and for other scenarios
such as wall demolitions.
8. Conflict of interest
We confirm that this manuscript has not been published else-
where and is not under consideration by another journal. None
of the authors have any competing interests in the manuscript.
All authors have approved the manuscript and agree with its
submission to the Special Issue of JCDE (Journal of Computational
Design and Engineering). We believe that the proposed system
and the result of this study will provide valuable information for
readers.
188 T. Fukuda et al. / Journal of Computational Design and Engineering 6 (2019) 179–188
Acknowledgement
This research has been partly supported by the research grant of
Nohmura Foundation for Membrane Structure’s Technology and
JSPS Grant Number 26-04368.
References
Asaumi, H., Nishina, H., Kei, T., Masui, Y., & Hashimoto, Y. (1994). Effects of foliage
plants on thermal environment and comfort inside room – Experiment analysis in
winter. Architectural Institute of Japan Planning, Environment and Engineering.
ASHRAE (2009). ASHRAE handbook – Fundamentals (I-P ed.). American Society of
Heating, Refrigerating and Air-Conditioning Engineers Inc.
Bahar, Y. N., Jeremie, L., Christian, P., & Nicolle, C. (2015). Integration of thermal
building simulation and VR techniques for sustainable building projects.
Conference 2013, 1–8.
Bille, R., Smith, S. P., Maund, K., & Brewer, G. (2014). Extending building information
models into game engines. In Proceedings of the 2014 conference on interactive
entertainment (pp. 1–8).
Bregon, N. F., Urrestarazu, M., & Valera, D. L. (2012). Effects of a vertical greenery
system on selected thermal and sound mitigation parameters for indoor
building walls. Journal of Food Agriculture and Environment, 10(3), 1025–1027.
Den Hartog, J. P., Koutamanis, A., & Luscuere, P. G. (2000). Possibilities and
limitations of CFD simulation for indoor climate analysis. In Fifth design and
decision support systems in architecture and urban planning – Part one:
Architecture proceedings (pp. 152–167).
Eastman, C., Teicholz, P., Sacks, R., & Liston, K. (2011). BIM handbook, a guide to
building information modeling for owners, managers, designers, engineers, and
contractors (2nd ed.). John Wiley & Sons.
Fanger, P. O. (1970). Thermal comfort. Copenhagen, Denmark: Danish Technical
Press.
Fanger, P. O., & Toftum, J. (2002). Extension of the PMV model to non-air-
conditioned buildings in warm climates. Energy and Buildings, 34(6), 533–536.
Fukuda, T., Mori, K., & Imaizumi, J. (2015). Integration of CFD, VR, AR and BIM for
design feedback in a design process. In Proceedings of the 33rd international
conference on education and research in computer aided architectural design in
Europe (eCAADe 33) (Vol. 1, pp. 665–672).
Fukuda, T., Zhang, T., & Yabuki, N. (2014). Improvement of registration accuracy of a
handheld augmented reality system for urban landscape simulation. Frontiers of
Architectural Research, 3(4), 386–397.
Gan, G. (1995). Evaluation of room air distribution systems using computational
fluid dynamics. Energy and Buildings, 23(2), 83–93.
Ghassemi, R., & Gerber, B. B. (2011). Transitioning to integrated project delivery:
Potential barriers and lessons learned. Lean Construction Journal, 32–52.
Goldsmith, D., Liarokapis, F., Malone, G., & Kemp, J. (2008). Augmented reality
environmental monitoring using wireless sensor networks. In 12th
International conference information visualization (pp. 539–544).
Gunnarsson, A. S., Rauhala, M., Henrysson, A., & Ynnerman, A., 2006. Visualization of
sensor data using mobile phone augmented reality. In 2006 IEEE/ACM
international symposium on mixed and augmented reality, Santa Barbard, CA,
2006 (pp. 233–234).
Hii, J. C., Zhou, Z., Karlekar, J., Schneider, M., & Lu, W. (2009). Outdoor mobile
augmented reality for past and future on-site architectural visualizations. CAAD
Futures, 557–571.
Holopainen, R., Milandru, A., Ahvenniemi, H., & Hakkinen, T. (2016). Feasibility
studies of energy retrofits – Case studies of nearly zero-energy building
renovation. Energy Procedia, 96, 146–157.
Holzmann, T. (2017). Numerics Mathematics Derivations and OpenFOAM(R) (4th ed.).
Leoben: Holzmann CFD.
Hosokawa, M., Fukuda, T., Yabuki, N., Michikawa, T., & Motamedi, A. (2016).
Integrating CFD and VR for indoor thermal design feedback. In Proceedings of
the 21th international conference on computer-aided architectural design
research in Asia (CAADRIA 2016) (pp. 663–672).
Juan, Y. K., Gao, P., & Wang, J. (2010). A hybrid decision support system for
sustainable office building renovation and energy performance improvement.
Energy and Buildings, 42(3), 290–297.
Klein, G., & Murray, D. (2007). Parallel tracking and mapping for small AR
workspaces. In: ISMAR 2007 Proceedings of the 2007 6th IEEE and ACM
international symposium on mixed and augmented reality (pp. 225–234).
Kovacic, I., Summer, M., & Achammer, C. (2015). Strategies of building stock
renovation for ageing society. Journal of Cleaner Production, 88, 349–357.
Laschewski, G., & Jendritzky, G. (2002). Effects of the thermal environment on
human health: An investigation of 30 years of daily mortality data from SW
Germany. Climate Research, 21(1), 91–103.
Lee, S., & Song, D. (2010). Prediction and evaluation method of wind environment in
the early design stage using BIM-based CFD simulation. IOP Conference Series:
Materials Science and Engineering, 10(1), 1–10.
Malkawi, A. M., & Srinivasan, R. S. (2005). A new paradigm for human-building
interaction: The use of CFD and augmented reality. Automation in Construction,
14(1), 71–84.
Matthias, B., & Veruna, C. (2015). CFD post-processing in Unity 3D. Procedia
Computer Science, 51, 2913–2922.
Montacchini, E., Tedesco, S., & Rondinone, T. (2017). Greenery for a university
campus: Does it affect indoor environmental quality and user well-being?
Energy Procedia, 122, 289–294.
Nakagawa, W., Matsumoto, K., Sorbier, F., Sugimoto, M., Saito, H., Senda, S., ...
Iketani, A. (2015). AR display system of temperature by camera tracking based
on 3d environmental model. The Journal of the Institute of Image Information and
Television Engineers, 69(4), 160–168.
Nielsen, P. V. (2004). Computational fluid dynamics and room air movement.
INDOOR AIR 2004, 14(Suppl 7), 134–143.
Nielsen, A. N., Jensen, R. L., Larsen, T. S., & Nissen, S. B. (2016). Early stage decision
support for sustainable building renovation – A review. Building and
Environment, 103, 165–181.
Pombo, O., Rivela, B., & Neila, J. (2016). The challenge of sustainable building
renovation: Assessment of current criteria and future outlook. Journal of Cleaner
Production, 123, 88–100.
Re, G. M., Kharshiduzzaman, M., Bordegoni, M., Bernasconi, A., Anodio, L. F., Comolli,
L., & Braghin, F. (2014). A mobile augmented reality framework for inspection
and visualization during fatigue tests. In ASME, Engineering systems design and
analysis, Volume 3: Engineering systems; heat transfer and thermal
engineering; materials and tribology; mechatronics; robotics (pp.
V003T10A017).
Rogerson, M., Gladwell, V. F., Gallagher, D. J., & Barton, J. L. (2016). Influences of
green outdoors versus indoors environmental settings on psychological and
social outcomes of controlled exercise. Int. J. Environ. Res. Public Health, 13(4),
363.
Rupp, R. F., Vasquez, N. G., & Lamberts, R. (2015). A review of human thermal
comfort in the built environment. Energy and Buildings, 105, 178–205.
Savioja, L., Mantere, M., Olli, I., Äyräväinen, S., Gröhn, M., & Iso-aho, J. (2003).
Utilizing virtual environments in construction projects. Special issue Virtual
Reality Technology in Architecture and Construction, 8, 85–99.
Schall, G., Wagner, D., Reitmayr, G., Taichmann, E., Wieser, M., Schmalstieg, D.,
Hofmann-Wellenhof, B. (2009). Global pose estimation using multi-sensor
fusion for outdoor augmented reality. In ISMAR 2009.
Schubert, G., Schattel, D., Tonnis, M., Klinker, G., & Petzold, F. (2015). Tangible mixed
reality on-site: Interactive augmented visualisations from architectural
working models in urban design. CAAD futures 2015 selected papers.
Communications in Computer and Information Science, 527, 55–74.
Srivanit, M., & Hokao, K. (2013). Evaluating the cooling effects of greening for
improving the outdoor thermal environment at an institutional campus in the
summer. Building and Environment, 66, 158–172.
Stam, J. (1999). Stable fluids. In SIGGRAPH 99 conference proceedings, annual
conference series (pp. 121–128).
Tan, C. L., Wong, N. H., Tan, P. Y., Ismail, M., & Wee, L. Y. (2017). Growth light
provision for indoor greenery: A case study. Energy and Buildings, 144, 207–217.
Thomsen, K. E., Rose, J., Morck, O., Jensen, S. O., Ostergaard, I., & Knudsen, H. N.
(2016). Energy consumption and indoor climate in a residential building before
and after comprehensive energy retrofitting. Energy and Buildings, 123, 8–16.
Uva, A. E., Cristiano, S., Fiorentino, M., & Monno, G. (2010). Distributed design
review using tangible augmented technical drawings. Computer-Aided Design,
42(5), 364–372.
Ventura, J., & Hollerer, T. (2012). Wide-area scene mapping for mobile visual
tracking. In: Proceedings of the IEEE international symposium on mixed and
augmented reality (pp. 3–12).
Voss, K. (2000). Solar energy in building renovation—Results and experience of
international demonstration buildings. Energy and Buildings, 32(3), 291–302.
Webster, A., Feiner, S., MacIntyre, B., Massie, W., & Krueger, T. (1996). Augmented
reality in architectural construction, inspection and renovation. In Proceedings
of the ASCE third congress on computing in civil engineering (pp. 913–919).
Yabuki, N., Furubayashi, S., Hamada, Y., & Fukuda, T. (2012). Collaborative
visualization of environmental simulation result and sensing data using
augmented reality. In 9th International conference, CDVE 2012 (cooperative
design, visualization, and engineering) (pp. 227–230).
Yabuki, N., Hamada, Y., & Fukuda, T. (2012). Development of an accurate
registration technique for outdoor augmented reality using point cloud data.
In: 14th International conference on computing in civil and building
engineering.
Yabuki, N., Miyashita, K., & Fukuda, T. (2011). An invisible height evaluation system
for building height regulation to preserve good landscapes using augmented
reality. Automation in Construction, 20(3), 228–235.
Yoshida, S., Ooka, R., Mochida, A., Tominaga, Y., & Murakami, S. (2000). Study on
effect of greening on outdoor thermal environment using three dimensional
plant canopy model. Journal of Architecture, Planning and Environmental
Engineering, 536, 87–94.
Zhu, Y., Ouyang, Q., Cao, B., Zhou, X., & Yu, J. (2015). Dynamic thermal environment
and thermal comfort. International Journal of Indoor Environment and Health, 26
(1), 125–137.
Zuo, W., Wetter, M., Tian, W., Li, D., Jin, M., & Chen, Q. (2015). Coupling indoor
airflow, HVAC, control and building envelope heat transfer in the Modelica
Buildings library. Journal of Building Performance Simulation, 9(4), 366–381.