D1.2.3 Initial Experiment Review Report v1.0 PDF

D1.2.
3 Initial Experiment Review Report

2013-10-29
Sandra Murg (JRS), Martin Lopez Nores (UVIGO), Jens Grubert (TUG), Alexandru Stan (IN2), Sergiusz Zielinski (PSNC), Yannick Naudet (TUDOR), Jose Manuel Jimnez (STT)
This document gives a detailed overview of the experiments added to EXPERIMEDIA in the first open call. Experiment goals, runs and results are described.
www.experimedia.eu
EXPERIMEDIA
Project acronym EXPERIMEDIA
Dissemination level: PU
Full title Experiments in live social and networked media experiences Grant agreement number 287966 Funding scheme Large-scale Integrating Project (IP) Work programme topic Objective ICT-2011.1.6 Future Internet Research and Experimentation (FIRE) Project start date 2011-10-01 Project duration 36 months Activity 1 Project Management Workpackage 1.2 Management of Experiments Deliverable lead organisation JOANNEUM RESEARCH Authors Sandra Murg (JRS), Martin Lopez Nores (UVIGO), Jens Grubert (TUG), Alexandru Stan (IN2), Sergiusz Zielinski (PSNC), Yannick Naudet (TUDOR), Jose Manuel Jimnez (STT) Reviewers Stephen Phillips (ITI) Version 1.0 Status Final Dissemination level PU: Public Due date PM19 (2013-04-30) Delivery date 2013-10-29
Copyright JOANNEUM RESEARCH and other members of the EXPERIMEDIA consortium 2013
EXPERIMEDIA
Table of Contents
1. 2. 3. Executive Summary............................................................................................................................ 5 Introduction ........................................................................................................................................ 7 Overview of first Open Call experiments ....................................................................................... 8 3.1. EX4: Digital Schladming.......................................................................................................... 9 3.1.1. Overall Goal .......................................................................................................................... 9 3.1.2. Technical Overview .............................................................................................................. 9 3.1.3. Ethics and Privacy .............................................................................................................. 12 3.1.4. Experiment Runs and Results ........................................................................................... 13 3.1.5. Value Impact Assessment .................................................................................................. 15 3.1.6. Dissemination ...................................................................................................................... 16 3.1.7. Conclusions.......................................................................................................................... 16 3.2. EX5: Novel mobile interfaces for situated media and interactive video MEDIAConnect...................................................................................................................... 17
3.2.1. Overall Goal ........................................................................................................................ 17 3.2.2. Technical Overview ............................................................................................................ 17 3.2.3. Ethics and Privacy .............................................................................................................. 19 3.2.4. Experiment Runs and Results ........................................................................................... 19 3.2.5. Value Impact Assessment .................................................................................................. 21 3.2.6. Dissemination ...................................................................................................................... 22 3.2.7. Conclusions.......................................................................................................................... 22 3.3. EX6: Remote and collaborative training for acrobatic sports - CONFetti .................... 22 3.3.1. Overall Goal ........................................................................................................................ 22 3.3.2. Technical Overview ............................................................................................................ 23 3.3.3. Ethics and Privacy .............................................................................................................. 25 3.3.4. Experiment Runs and Results ........................................................................................... 25 3.3.5. Value Impact Assessment .................................................................................................. 26 3.3.6. Dissemination ...................................................................................................................... 26 3.3.7. Conclusions.......................................................................................................................... 26 3.4. EX7: 3D Acrobatic Sports..................................................................................................... 26 3.4.1. Overall Goal ........................................................................................................................ 27 3.4.2. Technical Overview ............................................................................................................ 29 3.4.3. Ethics and Privacy .............................................................................................................. 30
Copyright JOANNEUM RESEARCH and other members of the EXPERIMEDIA consortium 2013 3
EXPERIMEDIA
3.4.4. Experiment Runs and Results ........................................................................................... 30 3.4.5. Value Impact Assessment .................................................................................................. 35 3.4.6. Dissemination ...................................................................................................................... 36 3.4.7. Conclusions.......................................................................................................................... 36 3.5. EX8: BLUE Personalized museum experiment ................................................................. 37 3.5.1. Overall Goal ........................................................................................................................ 37 3.5.2. Technical Overview ............................................................................................................ 37 3.5.3. Ethics and Privacy .............................................................................................................. 38 3.5.4. Experiment Runs and Results ........................................................................................... 38 3.5.5. Value Impact Assessment .................................................................................................. 40 3.5.6. Dissemination ...................................................................................................................... 40 3.5.7. Conclusions.......................................................................................................................... 41 3.6. EX9: REENACT experiment ............................................................................................... 43 3.6.1. Overall Goal ........................................................................................................................ 43 3.6.2. Technical Overview ............................................................................................................ 44 3.6.3. Ethics and Privacy .............................................................................................................. 45 3.6.4. Experiment Runs and Results ........................................................................................... 45 3.6.5. Value Impact Assessment .................................................................................................. 47 3.6.6. Dissemination ...................................................................................................................... 47 3.6.7. Lessons learnt for FMI/FIRE .......................................................................................... 49 3.6.8. Conclusions.......................................................................................................................... 49 4. Summary and conclusion................................................................................................................. 50
EXPERIMEDIA
1. Executive Summary
This report describes each of the experiments which were chosen to be included in EXPERIMEDIA through the first open call. The experiments were chosen through proposal submissions and a reviewing process by independent experts. The partners proposing the chosen experiments were then invited to join the EXPERIMEDIA consortium and conduct their experiments within 12 months. Six new experiments were invited into the project, adding eight new partners to the consortium. EX4: Digital Schladming, IN2, Schladming This experiment employs a platform with hyper-local information about the Schladming venue to investigate what parameters of a message can be changed to help engage target groups of stakeholders. Subsequently it helps the local stakeholders but also the region to understand what are the challenges of an approach with Future Internet technologies. EX5: MediaConnect, TUG + STI, Schladming The experiment investigates the influence of novel user interfaces on the Quality of Experience in accessing digital content situated in the real world. Specifically, it is assessed how experiences based on online data and services are influenced by a specific interface metaphor such as Augmented Reality, Virtual Reality, or digital maps. This experiment is meant to help to understand the potentials and challenges when employing innovative interfaces with situated content. EX6: CONFetti, PSNC, CAR This experiment tests an augmented reality 3D video conferencing tool for its usability in remote sports training. The system allows superimposing high resolution 3D models over the video feed by tracking an athlete's movement. With this a coach can watch over the accuracy of movements and can guide the athlete even from a remote location. EX7: 3D Acrobatic Sports, STT, CAR 3D Acrobatic Sports experiment focuses on high quality content production for assessment and improvement in gymnastic exercises by the use of inertial sensors for motion capture. The goal of this experiment is to record training sessions of gymnastics at CAR and automatically generate assessment data for helping the athletes improve their performance. Motion capture data recorded in the training sessions is shared between athletes and trainers using FMI products and services. EX8: BLUE, UoP, TUDOR, FHW The experiment explores the use of users visiting and cognitive styles in order to personalize their experiences inside a museum. Before and after a visit, visitors are invited to setup their cognitive profile and preferences, as well as to share their visits with others through a custom-made social network application, entitled My Museum Story.
5
EXPERIMEDIA
During the museum visit, smart routing recommendations for exhibits and content to see and personalized content descriptions are offered to them. EX9: REENACT, UVIGO, FHW The experiment engages groups of people who visit the FHW with an immersive collective experience that allows them to learn about a certain battle or war both from the inside, as reenactors, and from the outside, as historians. The activities can be supported by educators, guides and experts. The experience is organised in three stages: reenactments, replay and debate.
EXPERIMEDIA
2. Introduction
This deliverable provides a summary of the experiments that have used the facility from the first open call. The document provides the EC and the community with high-level information about what was done, a summary of results and the value of the facility. It was decided to adapt the focus of the deliverable to reflect the whole process of the experimentation phase and as such the document describes each of the first open call experiments with their concept, information about test runs and preliminary or final results. Furthermore the delay of the experiment runs caused this deliverable to be postponed after the first results would be available, to give a more complete view over the experiments in EXPERIMEDIA. As there was no follow-up deliverable planned in the Description of Work, this more thorough version of the originally planned "Initial Experiment Review Report" tries to amend this.
EXPERIMEDIA
3. Overview of first Open Call experiments

In EXPERIMEDIA the experiments testing the facility and baseline components developed in the project are the most important asset. To find promising experiments which would use the facility and advance the FMI while creating innovative user experiences a process of Open Calls was defined and used. These calls are used to get interested parties to submit proposals for experiments, which are reviewed by experts and after determining use and feasibility for the cause were chosen to become integrated parts of EXPERIMEDIA. The organisations proposing and subsequently conducting the experiments are added to the project consortium for the remaining project run time. The process of finding the experiments was as such: Defining open call text in the consortium EC reviewing the text Advertising the call in various media Managing proposal submissions Managing the review process Choosing the experiments and new partners
After consultation with partners, the call text was created detailing the types of experiment requested along with the constraints imposed by the technologies and venues. The call was published in three national newspapers and an international journal as well as the EXPERIMEDIA and Commission web sites. Further guidelines were made available on the EXPERIMEDIA website along with answers to frequently asked questions (FAQs) and a series of venue-specific open call information teleconferences were held with 50 organisations attending. Twenty-nine proposals were received before the call deadline and were efficiently evaluated over the space of 1 week by a pool of 8 independent experts. The contract amendments were quickly made in time for the 8 new partners to formally join the project at the start of October.
Experiment EX4: Digital Schladming EX5: MediaConnect EX6: CONFetti EX7: 3D Acrobatic Sports EX8: BLUE EX9: REENACT Partner IN2 TUG, STI PSNC STT UoP, TUDOR UVIGO
Table 1: 1st Open Call Experiments
Venue Schladming Schladming CAR CAR FHW FHW
In the following chapters the experiments are explained in detail.
EXPERIMEDIA
3.1. 3.1.1.
EX4: Digital Schladming Overall Goal
DigitalSchladming provides the necessary services to Schladming to engage with guests visiting the region. The experiment addresses the needs of visitors for a wide range of information access, from practical issues during their stay, e.g. what to do tomorrow or where to find the best Italian restaurant, to sharing their experiences. DigitalSchladming is based on own work and results of previous projects and the SCC an AVCC baseline components of EXPERIMEDIA. The proposed experiment is centered around an information-rich, web-based media channel for the entire region, the Schladming Pinboard, and a personal hub for media contents of each end-user, MySchladming. The two systems are seamlessly interconnected, allowing the end-users of MySchladming to effortlessly publish the content they want to the Schladming Pinboard. Together the two systems are able to create a hyperlocal temporally bound community. Since information access is not only restricted to guests visiting the region, local citizens also benefit in the same way from DigitalSchladming. In addition to this, the project provides local businesses with a new digital channel to reach their target audiences and offer their services. Businesses not only can engage in advertising targeted to consumers but also listen to needs or requests of consumers and react accordingly. Thus, we believe that DigitalSchladming can evolve into a marketplace of services for both visitors and citizens and this will have a very large impact for the Schladming region. Finally, destination managers can take advantage of the curational aspects of DigitalSchladming and promote the region as a whole, with the aim to deliver the best experience to visitors and guests and so make them return in the next season.
3.1.2.
Technical Overview
The figure below shows the different components of the DigitalSchladming architecture:
EXPERIMEDIA
Figure 1: Schladming Pinboard Architecture
3.1.2.1. Schladming Pinboard The Schladming Pinboard was inspired by the various analogue pinboards found in the lobbies of hotels, which contain both general information as well as flyers of special events. We set out developing the Schladming Pinboard with the aim to build a public digital space which aggregates data and media feeds (images, videos, audio files and documents) from a variety of sources, including popular social media networks currently in use by Schladming stakeholders (e.g. Facebook, YouTube) as well as channels especially set up for this experiment. The Schladming Pinboard becomes a unified media channel with powerful faceted search capabilities. Tags and categories are extensively used in order to allow the user to find the right information easily. The Schladming Pinboard underwent a number of changes as user-feedback has been collected during the experiment lifetime. This has allowed the back-end infrastructure to develop new functionality (e.g. number of data sources fetched, type of metadata used, etc) as well as evolve
EXPERIMEDIA
the user interface to become more flexible, interactive and visually engaging (responsive and with a mobile view, dynamic masonry, direct links, etc). The figure below shows the final look and feel of the Schladming Pinboard.
Figure 2 - View of Schladming Pinboard when filtering by content type "Official" (to show only content belonging to official channels such as Planai, Schladming-Dachstein, etc)
The Schladming Pinboard presents content coming from the following sources: Facebook Twitter YouTube Flickr Instagram Official Websites MySchladming Followtheplace
3.1.2.2. MySchladming MySchladming is not a separate web application from the Schladming Pinboard. The two are very well interconnected through the backend server. From the user interface of the Schladming Pinboard a user is one click away from MySchladming (provided that he is already logged in). MySchladming provides the visitors with an easy-to-use platform for storing and sharing the digital memories of their vacation in Schladming. MySchladming can handle a variety of media files: documents, audio, images, video. At upload time the user specifies all the additional metadata which will be explicitly added to the file. This metadata will be used for content retrieval.
EXPERIMEDIA
A MySchladming account can belong to one user or a group of users, which is particularly handy for families since they now can store and manage all of their Schladming holiday memories in one single repository which lives in the cloud and can be accessed from anywhere. If the account is shared than it is up to the users to arrange that all people in the group have the necessary credentials to login. Although the look and feel is similar to that of the Schladming Pinboard (i.e. the main elements of the interface are still there), some of the functionality is different. More specifically, each content card now shows only the image and title, alongside the option to Edit, get More information or View the content item itself. The Facet panel now contains different facet categories: Type (image, document, video, audio), Collection (Private, Pinboard) and Popular keywords extracted from the title. The choice of facets reflects the fact that in MySchladming the users sees only content he himself has uploaded on the DigitalSchladming service. Figure 3 shows how the interface looks to the user. If a user will like to share a content item with the entire community it is only necessary to Edit the Collection facet category and changes are immediately effective.
Figure 3 - The MySchladming interface
3.1.3.
Ethics and Privacy
Any internet user can access the Schladming Pinboard at the following address: http://onmeedia.com/apps/Schladming. For MySchladming minimum personal information is necessary for registering a new account: email address, a first name and a last name. It must be noted that users are free to use nicknames instead of their real names. Once the user has selected an appropriate password, the registration form can be submitted. An activation email will then be automatically sent to the email address
EXPERIMEDIA
provided in the form. To complete the registration process the user has to click on a link provided within the email. Once this is done the user is successfully registered. At the bottom of the DigitalSchladming page there is a clear link to a legal section which explains in detail the Privacy and Terms and Conditions for using the Digital Schladming service: http://onmeedia.com/apps/schladming/legal.html . It must be noted that this section has received considerable care and was prepared in collaboration with EXPERIMEDIA IPR experts. Of note is the fact that the language used is more accessible to a regular person than the one generally found in such section of online services. The aim here was to ensure that users understand the privacy and terms and conditions that apply when they use the service and take part in the experiment. We aimed to ensure that users do not need to disclose sensitive information and that we protect their privacy. To this end only anonymised data was recorded and analysed. For the registered users we stored their email addresses and provided nickname (as described in the section about MySchladming).
3.1.4.
Experiment Runs and Results
For a successful deployment DigitalSchladming required a close cooperation with local stakeholders, not just those within the EXPERIMEDIA consortium but also those from the outside. To facilitate this cooperation it was suggested in the early phases of the experiment to have a continuous experiment rather than to employ discrete experiment runs at pre-defined times. This is why after the initial public release of DigitalSchladming v1.0 on 25/03/2013 we have kept the system running. In this way we were able to gather a significant amount of data. Improvements to the live system were done on a rolling basis after being preliminarily tested on a development server. To better understand how the users are interacting with the system we have additionally conducted user evaluations on location. In total we had two such sessions in Schladming. The first one was on the week of 25 to 29th of April 2013, which coincided with the first release of the system and the launch campaign we conducted with this occasion. We conducted presentations and gathered direct feedback from Schladming Stakeholders, including local businesses. The second session was during the MidEurope Festival in Schladming (09/07 14/07/2013) and coincided with a joint activity of all of the Schladming experiments to raise awareness and engage in on-site data-gathering. The version of DigitalSchladming which was presented at this second on-site session was much improved over the initial version released over two months before. Again feedback was gathered from local stakeholders. The feedback received from the local businesses towards the Digital Schladming service and how they can use it to reach new customers has been quite mixed, with most responses being interested in the application, while 1 respondent was not interested and some being very interested. These responses are in close relation to the digital literacy level and the extent to which they do already use digital technologies for their business. The usage statistics of MySchladming (which have been much lower than originally expected) supports the conclusion that those interviewed which reported an interest in the application (but were not in either extreme of the scale) had difficulties to differentiate between the other digital tools made available in the Schladming region (such as Schladming App, Pinpoint App, etc). Thus, a clear
EXPERIMEDIA
message outlining in plain language the impact for their business and the usage scenario of the application is paramount for any further application to deployed and experimented in Schladming or similar setting. From the usage data collected after the initial public release of the application, we could observe a small difference in the number of users of the application based on the season. While between March and May there have been about 190 unique users per month, the summer months following had on average approximately 130 unique users. This drop in the number of unique visitors comes in spite of the gradual improvements brought to the application over time. The reason behind this can be explained by realising that the region of Schladming has many more tourists in the winter season than in the summer one. As expected, spikes in the number of users of the application were correlated with the promotional campaigns conducted on site during the initial launch and the MidEurope event. On average the users spent about 3 minutes on each visit to the DigitalSchladming application. Throughout the month the number of unique users is also varying on a day by day basis as can be seen in the figure below:
Almost three quarters of the users came to the application by directly accessing the URL, while about a fifth of the number of users were referred to by other websites, most notably Wikipedia. The rest of the users found the application via a search engines. It becomes apparent from this that disseminating the service on site (e.g. via flyers, posters, press, etc) plays an important role in increasing the usage of the web-application during experimentation phase, when integration of the service is not done in tone of the main tourism websites of the region, such as www.schladming-dachstein.at Data has also been collected about the geographical regions from which the users are coming from. It is interesting to see that in this respect there is almost an equal split between users who are coming from the region of Schladming and those who are from the outside. This indicates that the Schladming Pinboard might be used by a good number of users when they are preparing their holiday plans, for assessing the touristic destination. Further research is however needed in order to investigate the extent to which this takes place and the kind of influence that the application has on their decisions for a holiday location.
14
EXPERIMEDIA
3.1.5.
Value Impact Assessment
The Digital Schladming experiment interacts with a wide variety of stakeholders, with very different inters and needs. It realises however to create an impact for each of them. Thanks to Digital Schladming Tourists can get a quick overview of what is happening in Schladming at a given point in time. This is something that was not available before, when to get a similar overview one would have had to manually go to and check each of the relevant information channels such as websites, Facebook pages, twitter users, Youtube channels, Flicker or Instagram, They would have to repeat this tedious process every time thy would want an update. Naturally this assumes that they knew which information channels are relevant for the region of Schladming (which is rarely the case with tourists who are new to a region) and that they have a user's account for those respective channels since full access is generally granted only to logged-in users. What is more, now that they have all the information in one place, automatically being updated and presented in a visually rich way, they can more easily get a grasp of what is happening in the region at the moment and even search (across all channels) for a specific keyword or browse the content based on certain criteria (e.g. show only content from official sources). For the venue the DigitalSchladming web-application can help strengthen the brand of the region and provide better online visibility and reach. Currently discussions are underway to integrate the DigitalSchladming web-application in one of the official Schladming stakeholder websites. In this way visitors who land on the tourism site will no longer be driven somewhere else to check for more updated information and social interactions. The community managers for Schladming tourism will effectively be able to reclaim the relevant content from the numerous social media channels and bring it back to the tourist website of the region. More, since we have seen that a good number of DigitalSchladming users are accessing the web-app from outside the region, it becomes also a new avenue for attracting new visitors since those who are planning a future holiday are now able to see that Schladming has a vibrant community with many options for having a great time. For the research community and the experiment has provided a number of interesting insights into how well hyper-local technology currently performs. We have seen that the presentation of aggregated contextual information based on location in a tourism scenario (e.g. Schladming Pinboard) is indeed perceived as an advantage by its users. We have also learned that an additional service for hyperlocal sharing of content (e.g. MySchladming) has a slow adoption rate, and is currently too soon to introduce it as a real application. IN2 has benefited greatly by conducting this experiment within the EXPERIMEDIA framework. Firstly through the close collaboration with the venue we have gained valuable insights into what is perceived as useful by destinations and tourists. The technology components of EXPERIMEDIA have helped us set-up the web-application quickly. Thanks to this experiment IN2 is now better positioned to enter the market of software solutions for the tourism industry, being able to show a successful demonstrator that makes use of the latest technologies that the Future Media Internet has to offer.
15
EXPERIMEDIA
3.1.6.
Dissemination
Dissemination activities have begun with the creation of a brand and visual identity for DigitalSchladming. It was important to establish a brand which would be easily recognisable to the different stakeholder while being in line with the overall brand of Schladming tourism. The figure below shows the final logo.
Figure 4 - Digital Schladming logo
Together with the Schladming venue representatives, a comprehensive dissemination campaign has been prepared in order to inform all local stakeholders about the launch of the DigitalSchladming service. Both online and offline channels have been targeted. For the online channels, an announcement of the launch has been made on the official Schladming webpage, the official Schladming2030 webpage and official Facebook pages. Offline, a series of flyers have been created for different stakeholders, i.e. English speaking visitors and general public, German speaking visitors and general public, local businesses (in German), and spread at different locations throughout Schladming. Similarly, several posters have been created and displayed through channels of the Schladming municipality. Moreover, a press release has been communicated to the local newspaper, Der Ennstaller. IN2 has disseminated DigitalSchladming and the EXPERIMEDIA project at Technology Solutions for Tourism 2013 - Where Technology and Tourism collaborate, which was held in Edinburgh in March 2013 and was organised by ETAG (Edinburgh Tourism Action Group). During the event we were able to inform the tourism stakeholders and technology providers present about DigitalSchladming and EXPERIMEDIA via flyers and face-to face conversations at our exhibition booth. Finally together with the driving experiment and MediaConnect, Digital Schladming was presented between 7th and 14th of July at the MidEurope festival in Schladming, one of the most recognized wind festivals in the world. During MidEurope a booth and a poster was set-up at the entrance of the main venue of the event. Flyers were also distributed individually to various guests who were attending the many outdoor events throughout the town. Finally the advancements with DigitalSchladming were presented in another meeting with Schladming venue stakeholders.
3.1.7.
Conclusions
DigitalSchladming has explored how the new trend of hyper-local services can be used in tourism scenarios. To this end a web-based application has been built which aggregates contextual content relevant only for the region of Schladming and provides its users with an
EXPERIMEDIA
intuitive interface to browse and search for content. Moreover, it allowed everyone to upload and manage personal content during their stay in the region. The SCC and AVCC components of EXPERIMEDIA have been used while a close contact to the venue was invaluable for developing and launching the service. The web-application has been trialled with a large number of users over more than five months and was shown to provide advantages for both tourists and the venue itself. The DigitalSchladming web-application is still running and there are currently talks for integrating it into the official touristic website of the region.
3.2.
EX5: Novel mobile interfaces for situated media and interactive video MEDIAConnect Overall Goal
3.2.1.
The MEDIAConnect experiment centres on enriching tourists' experiences when interacting with local services and products through novel mobile interface technologies like Augmented Reality and Interactive Video. Specifically, a focus of the MEDIAConnect experiment is enhancing the experience of existing products and services through novel mobile interfaces rather than providing a complete service itself. The goal of the technology for visitors must be that they feel they have better used their time and resources while in the region, e.g., they found out about and could attend locations and events of interest to them or activities at Schladming were more accessible (they could preview the activity, reserve in advance, go there at the moment they were close by, etc.). Objectives of the MEDIAConnect experiment are to capture the user experience when interacting with local services, products and events in Schladming made accessible through novel interfaces including Augmented Reality and Interactive Video on the users' mobile devices. Furthermore, assessing the user experience of different mobile interfaces should have multiple benefits for the local business stakeholders in Schladming.
3.2.2.
Technical Overview
The MEDIAConnect experiment centers around a mobile Augmented Reality application and a interactive video player which are described next. 3.2.2.1. Mobile Augmented Reality Client The mobile Augmented Reality (AR) client is a computer vision based AR system implemented with the Unity3D game engine and the Qualcomm Vuforia AR toolkit. Implementing the client with this software framework enables both rapid prototyping and a deployment to both major mobile platforms iOS and Android, allowing for a potentially widespread take up by smartphone and tablet users. The application (see Figure 5) is based around the idea of enabling users to quickly browse digital content sources linked to locations depicted on a poster by physically navigating the information space. Further detailed exploration of the individual content items (like photos or videos) is then supported in 2D views.
17
EXPERIMEDIA
Figure 5. The mobile Augmented reality application.
3.2.2.2. Interactive Video Player The interactive video player (see Figure 6) is based on the open source LIME player1 (making use of the HTML video tag, video.js, JQuery, CSS3, JSON2, backbone.js, underscore.js, RDFQuery and VIE library) and runs in the latest version of any of the main browsers. The (desktop) Web based version which responds to mouse events has been adapted to both run in a smartphone/tablet (with touch interaction) as well as over Google TV (reacting to remote control events). The player incorporates support for the W3C Media Fragment syntax that should allow video to be accessed not as an entire media resource but in terms of a temporal and/or spatial part thereof. As the video plays, Javascript code checks for annotations on the next active video segment, and enables access to additional content when it is relevant to the concept annotating that segment via a plugin and widget architecture. Annotations refer to Linked Data resources and the ConnectME framework has collected links to content relevant to those resources using the Linked Services Infrastructure. The hypervideo player has a core that sustains the video playback mechanism and connects to the ConnectME Framework to retrieve the annotations in an initialization phase. A set of plugins is then attached to the core, each of which is specialized in recognizing a certain type of annotation resource.
The LIME player: https://github.com/tkurz/lime
18
EXPERIMEDIA
Figure 6. The interactive video player.
The employed videos are loaded from the EXPERIMEDIA testbed component AVCC. While we aimed to integrate the ECC component for data analysis technical limitations of the component did not allow us to complete integration with our web-based (Interactive Video) and Unity-based (Augmented Reality) software in time.
3.2.3.
Ethics and Privacy
We identified metrics which we would collect throughout the experiment. They were mostly based on device data and subjective measures in written questionnaires. KU Leuven and the Austrian privacy commission approved our data collection methods. For the experiments participants were informed using a written informed consent form
3.2.4.
We report on the experiment runs of the Augmented Reality and the Interactive Video application individually. 3.2.4.1. Augmented Reality For the first run we designed a within-subjects study comparing the effects of Augmented Reality and 2D Map interaction on user performance and user experience in a locator task on a physical map in a public space. The task was to find the single lowest priced restaurant among 16 candidate locations on a ski field map. The study was conducted outside the Planai ski station in Schladming during the skiing season in March 2013. 18 volunteers from around the area participated in the study. The results of the first run indicated that the physicality of the Augmented Reality interaction did not suffice to engage users more than a traditional screen based interaction and did not show performance benefits. However, previous work indicated that increasing the workspace size can lead to an improved performance of Augmented Reality compared to interfaces which do not utilize visual context. Hence, we designed the second run to investigate how varying workspace sizes would affect the performance of AR vs. 2D map navigation. We chose a comparable experimental design to the first run. As our experiment was focused on utilizing several map sizes it could not be conducted in Schladming itself but was executed in a laboratory environment. Confirming our first study it
EXPERIMEDIA
appeared that 2D map outperforms AR for comparably small workspaces. No benefits could be found for task completion time and subjective measures. For the medium and large workspaces we found equivalent task completion times. Subjective ratings and feedback in the post-hoc interviews indicated potential usability benefits for AR. Participants reported less workload for the larger workspaces for AR. For the large workspace AR had a significant usability rating. Better overview and a better sense where near-by candidates are located were commonly reported by participants as reasons for the better usability. 3.2.4.2. Interactive Video For the first runt the video player has as unique source a promotional video of the Schladming area during the winter season. The video shows, alongside its frames, an interactive menu with items sliding over the video frame containing additional information about the currently mentioned concepts within the video. There were 13 participants (7 males, 6 females) to the interactive video first run experiment, 5 of which attended the testing sessions in Vienna, Austria and 8 tested the interactive video in Schladming, Austria. The main scope of the first run was to get feedback about the usability of the Interactive Video player. The testers welcomed the idea of interactive video and the majority of them felt comfortable using such a technology. There was an open debate regarding the relevance of the annotations. Testers from Schladming pointed out that the video was boring for them as it described an area very well known to them, but the participants in Vienna showed strong interest in the video and found the widgets helpful. Although there was an extended interest, shown by all participants, regarding the widgets targeting local business, the iPad browser restricted scrolling within the iframe and this created a well noted confusion. The YouTube videos proved to be a comfortable integration as a video within a video type of display maintained the flow of viewing. The interaction with the maps was known by all participants a priori. We used the Google maps engine to render interactive maps, weather information and routes. Info widgets had a high rate of accesses but the time spent on them is relatively low. Following a re-design of the user interface the second run was conducted online, promoting a link to the interactive video player with the video source Planai News Summer activities in Schladming (http://mediaconnect2.connectme.at). Visitors following our call to test interactive video technology via Social Web channels (distributed over Facebook and Twitter to friends and followers of the experimenters and organisation STI International) used the Web link to access and watch a 2 minutes 53 seconds video within our hypervideo player. 25 people participated in the interactive video second experiment run. The experiment was run internationally and persons from 14 countries took part in this experiment. More than 48% of the participants visited the interactive video more than once. As the second run was conducted purely on the internationally, no pre-, during-, or post-experiment questionnaires were used. We studied purely the behaviour of the testers and the way they interact with our interactive video player. The interactive video was accessed using several platforms, and due to that, some browser incompatibility and errors of access occurred. Figure 12 shows a chart of platforms that the player was accessed from and their compatibility rate. The second run contained more relevant annotations. The annotation number was reduced to fit a more comfortable play-out and their life span was increased so that the testers could easily notice the widgets. We believe that the application was tuned to fit a more commercial use case in which the particular interactive video
EXPERIMEDIA
was enriched with the purpose of adding cultural and marketing value to the Schladming Planai touristic region. The overall application usage closely resembles a real life product usage of this sort. Participant demographics was widely spread around the globe, but the majority of accesses were made in Europe. Not all participants interacted with the video and not all participants that interacted watched the video until the end. A higher interest rate can be achieved in the future by better assigning annotations on the video and creating new types of widgets. Nevertheless, we demonstrated that in a real life context, our technology was interesting for a significant niche of persons. The interaction behaviour of the participants was not uniform, nor guided by tasks, therefore widget accesses varied widely. We can conclude that the factual information widgets and the geographical widgets were the most popular. They were followed closely by the booking widgets, although they were not pointing to the real business owner and participants were notified that the booking is not possible. We realize that expanding such booking widgets and creating such connections between tourists and business owners offers a lot of business potential. Video widgets have a lower average access, as opposed to the first run in which they had a higher interest. This may be due to the fact that we used a semantic video search engine which did not provide manually selected videos. A future development to improve relevant results from the video search engine will add to the value of the overall interactive video experience.
3.2.5.
The Interactive Video has the potential to complement traditional video and raise additional interest in the region. Video material that is produced for advertising purposes can be easily reused, annotated and thus transformed into Interactive Videos to create a further dissemination channels for Schladming. The usefulness of computer-vision based AR for non-leisure tasks as studied in our experiments still needs further investigation. The main contribution of our studies was to understand better the strengths and weaknesses of mobile Augmented Reality and compare them with 2D map interfaces given contemporary sensing technologies on small screen handheld devices. It was interesting to observe that mobile Augmented Reality has potential benefits with large workspaces, and that participants seemed to have a better overview. It would be interesting to purse this work with large workspace on more complex tasks where survey knowledge is critical to the success. Indeed, we are in the process of planning a follow-up in-the-wild study, which will take place outside EXPERIMEDIA, for AR with large hiking maps in Schladming, together with local stakeholders. Baseline technologies and venues provided the potential for meaningful assessment of our technologies. However, venues such as Schladming, with many stakeholders and diverse users, need to be better understood and integrated to maximize the potential for assessments on-site. Also, further optimizations in interfaces to baseline technologies will help to streamline the deployment process of FMI prototypes. In particular we were successful in integrating baseline technologies provided as Software a Service (AVCC) and experienced challenges in integrating other technologies such as the ECC, eventually needing to use fallback solutions.
21
EXPERIMEDIA
3.2.6.
Dissemination
The MEDIAConnect Augmented Reality prototype was presented at the Future Internet Assembly 2013 in Dublin. A paper about Augmented Reality usage was submitted to a major Human Computer Interaction conference (ACM CHI 2014), notification is expected in December 2013. In addition a MEDIAConnect poster was presented at the Mid Europe music festival in Schladming in June 2013.
3.2.7.
Conclusions
The MEDIAConnect experiment focused on enriching users experiences when interacting with services and products through the provision of novel mobile interface technologies like Augmented Reality and Interactive Video. The results of the experiments gave insights into the potentials and challenges of mobile Augmented Reality and Interactive Video technologies. To reflect about our initial research question: Can the QoE in interacting with situated content in mobile contexts be enhanced through novel interfaces? We can answer: yes, but advantages of tested technologies are not as clear as initially expected and need further studies. Better integration between stakeholders (experimenters, venues, baseline technology providers) will further strengthen the impact future experiments can have.
3.3. 3.3.1.
EX6: Remote and collaborative training for acrobatic sports - CONFetti Overall Goal
The CONFetti experiment aimed to investigate the possible applications of FMI technologies in the improvement of the sports training process employed in the CAR venue. The technologies that were utilised included HD videoconferencing, stereoscopy, motion tracking and augmented reality. The experiment made use of a prototype system allowing the coach to hold a training session without the need for his presence in the training hall. He connected remotely with his protgs located in the hall using a high definition (HD) videoconferencing tool. The system enabled him to observe the athletes performing their gymnastics routines and, afterwards, to give them advice while reviewing an instant replay with slow motion capabilities. Additionally, the coach was able to review and present archival footage of athletes recorded with stereoscopic (3D) HD cameras. This footage also contained a 3D model of a human body reflecting the athlete's movements collected via motion tracking. The stream with the superimposed model was treated as one of the sides of the videoconference for both the athletes and the coach to view. The experiment's goals from the venue's point of view were to evaluate the possible benefits offered by the various FMI technologies used. The training process could get improved in the following ways: The availability of coaches for sessions is incremented as they can be held even when the coach is not present at the venue. The convenience for the coaches is raised thanks to the possibility of holding sessions from remote locations.
22
EXPERIMEDIA
The coach has access to additional motion tracking and sensor data concerning the athletes' performance, on which he can base his decisions. The transfer of knowledge and instructions is more efficient with the use of archival footage and 3D model teaching aids. The use of new technologies like augmented reality or stereoscopy may appeal to young athletes and make the sessions more interesting and, in turn, more beneficial.
The experimenters carried out the experiment in order to investigate the following issues: The feasibility of installing and utilising a setup for stereoscopic HD videoconferencing and augmented reality in a real-life training facility User acceptance level (concerning both coaches and athletes) of the FMI functionalities provided by the system deployed in a sport training setting Performance of a platform combining HD videoconferencing with stereoscopic video and remotely rendered 3D models generated basing on motion tracking
3.3.2.
Technical Overview
The CONFetti platform combines a number of FMI technologies and was integrated with some external components, all of which are mentioned below. 3.3.2.1. HD Videoconferencing This component's main task is to enable a stereoscopic HD videoconferencing connection between the athletes in the training room and the remote coach. Additionally, it allows the coach to review and present the archival footage of previous athletes' performances. There are two instances of the videoconferencing client, one running on a machine at the coach's site, the other in the training hall at CAR. The videoconferencing solution adopted for the CONFetti experiment is based on HDVIPER , an open and scalable HD videoconferencing platform developed in the course of the CELTIC initiative with a substantial participation of PSNC. The aim of the HDVIPER project was to provide a converged platform for High Definition videoconferencing (point-to-point and multi-point transmissions), to be integrated in a PC environment (pre-commercial prototype) and to be tested in residential VC, healthcare, business VC and distance education scenarios. Thanks to its extendable architecture the platform's functionality can easily be broadened with additional services apart from HD video and audio connectivity. The architecture is based on SIP (Session Initiation Protocol), which is very widespread in Internet multimedia applications. Its popularity makes possible the platform's compatibility with a wide array of solutions including software phones, smart phones and videoconferencing terminals. HDVIPERs architecture is compatible with various client applications, but for the purposes of the experiment one solution had to be selected and appropriately extended. The chosen software is Minisip, an open source SIP User Agent developed by the Royal Institute of Technology (KTH) in Stockholm and a community of developers including PSNC employees. 3.3.2.2. 3D The 3D functionality utilised in CONFetti is based on stereoscopy. A number of dual-lens cameras are used to acquire footage of the athlete performing the routine, the coach and the
EXPERIMEDIA
athletes gathered by the training hall VC terminal. The stereoscopic signal is transmitted through the videoconference in the Side-by-side format and displayed on 3D enabled TVs. The experiment participants are required to wear dedicated glasses in order to view the 3D video feed. Additionally, a 3D transcoder component was developed with the ability to convert the 3D signal to other formats than SbS. 3.3.2.3. Motion Tracking The motion tracking functionality in the CONFetti experiment was realised in cooperation with STT, the EXPERIMEDIA partner responsible for the 3D Acrobatic Sports experiment. The first run of the experiment was used as an opportunity to record gymnastic performances and, in parallel, gather motion tracking data of the same performance, allowing to create 3D models of the athletes bodies animated using this data. During CONFettis first run inertial Bluetooth sensors were utilised, whereas for the second run they were replaced by reflective markers. 3.3.2.4. Visualisation The visualisation component is used to render the 3D models that are superimposed on the archival video material. The input data for the animation rendering process are models and sensor events acquired by motion tracking done by the STT experiment. The software used for this component is the Vitrall visualisation system designed and implemented at PSNC. Vitrall is a distributed, web based visualisation system that is capable of remote rendering of complex 3D content coming from various sources, e.g. scientific data or multiple 3D model formats. It is designed to efficiently utilise multi-GPU and CPU-GPU hybrid server installations. Thanks to highly efficient remote server rendering even a less capable client is able to efficiently display and manipulate large 3D datasets in real-time. The system can also be used in collaborative environments, where many users can simultaneously interact, modifying the 3D content in real-time. Vitralls high-performance remote visualisation capabilities have been successfully demonstrated at various international conferences. 3.3.2.5. Integration with the Kinect Cycling Experiment The CONFetti platform was integrated with the system utilised by CERTH in the Kinect cycling experiment. This allowed holding a 3D HD videoconference between the cycling training spot and a remote location with statistical data from CERTH's system superimposed onto the stereoscopic videoconference stream. 3.3.2.6. Experiment Content Component The ECC's role is to gather monitoring QoS data from the other components that register as sources of such data. Since all the software components of the CONFetti setup are written in C++, a C++ client for the Experiment Content Component was needed. An API delivered by the component's creator enabled the experimenters to implement the required client functionality in the system and use it to gather usage statistics during the second run of the experiment. 3.3.2.7. Audio Video Content Component The role of AVCC in the CONFetti experiment was to serve as a video repository. Videos of athletes performing the gymnastic routines with 3D models superimposed were uploaded to an
EXPERIMEDIA
AVCC instance and can be streamed to the Minisip videoconferencing client in order to review them during the training session, giving the coach a possible source of training aids.
3.3.3.
Ethics and Privacy
The key ethics and privacy issues to consider for the CONFetti experiment were the storing and processing of video material including coaches and athletes and the storing and processing of anonymised questionnaires. After a few rounds of talks and gathering data by KU Leuven, a controller processor agreement between CAR and PSNC was drafted to solve these issues.
3.3.4.
Two runs of the CONFetti experiment were performed. The first one, done on May 15-16th 2013, included an HD 3D videoconference with instant replay and frame-by-frame functionality. Archival materials recorded in January 2013 were also available for the coach to use, although they did not yet contain 3D models of the athletes' bodies. The opportunity was used to record gymnastic performances in parallel with tracking the athletes' motions performed by STT. This allowed to subsequently render the 3D models for use in the future. Additionally, all the experiment's participants were asked to fill questionnaires concerning their views on the system, its different functionalities and the difference it made in the training session.
Figure 8. A coach and an athlete reviewing a performance during the first run of CONFetti
The second run of the CONFetti experiment was performed on 16-18th September 2013. This time, apart from exposing the participants to the functionalities present in May, the 3D models rendered basing on motion tracking data collected by STT were used. Additionally, the CONFetti platform was integrated with the system utilised by CERTH in the Kinect cycling experiment. The opportunity was also taken to record video materials for the purpose of promoting the experiment and project. The analysis of the questionnaires and the QoS data measured during the experiment runs shows that the system is suitable for application in a training venue, both performance- and functionality-wise. The feedback that was gathered also contains useful remarks concerning possible enhancements and fine-tuning of the systems parameters.
25
EXPERIMEDIA
3.3.5.
The experiments results show that the deployment of a system offering CONFettis functionalities in CAR would be beneficial for all the stakeholders involved. The athletes found it fun and interesting to use and also stated that the instant replays in slow motion and 3D help them review their performance and understand the coachs remarks. For coaches, the additional functionalities, the high quality of the VC signal and the stereoscopy make the remote training session a valid alternative to personal interaction, allowing them to save time and be more flexible in scheduling. For the venue, the system could allow to decrease the costs of the training program and increase the availability of coaches, which is especially important in the times of austerity.
3.3.6.
Dissemination
A paper and presentation on the CONFetti experiment titled Stereoscopic videoconferencing with augmented reality in technology enhanced sports training was prepared for the eChallenges 2013 conference. Presentations on the experiment and EXPERIMEDIA project were also performed during the i32013 conference and FI-PPP InfoDay and Brokerage Event in Pozna. The technology involved in the project was utilised in a stand at Researchers Night 2013. Additionally, a promotional videoclip about the experiment was created.
3.3.7.
Conclusions
The system created during the CONFetti experiment proved to be useful for sports training applications. The feedback gathered from both the athletes and the coaches verifies that, while additionally giving some interesting remarks about possible enhancements. The venue also found it beneficial to make use of the systems possibilities in the training process. The tests ha ve also proven that it is feasible to set up such a system in a real-life training hall and that its performance is satisfying for the purposes evaluated in the scenario.
3.4.
EX7: 3D Acrobatic Sports
Under the umbrella of the EXPERIMEDIA project the 3D Acrobatic Sports experiment focuses on high quality content production for assessment and improvement in gymnastic exercises by the use of motion capture technologies and biomechanical analysis. The goal of this experiment is to record training sessions of gymnastics at CAR Venue and automatically generate assessment data for helping the athletes improve their performance. These 3D motion capture data is also used to compute metadata which is synchronized and saved with the athletes motion in order to provide a valuable 3D graphics and augmented reality experience. The experiment makes use of the connectivity and storage facilities available at CAR Venue. The experiment makes emphasis on the quick delivery of data to the athletes and trainers. To this end it makes use of laptop and tablets in order to collect data from motion capture devices, either inertial sensors or optical motion capture systems. The motion capture data is processed in the host computer and instantaneous feedback is provided to the users even in real-time. In the practical execution of the experiment both technologies (inertial and optical) have been used in training sessions carried out in real conditions. The use of motion capture devices did
EXPERIMEDIA
not impose any constraint to the motion of the athletes due to the reduced size and weight of these devices.
Figure 7: Inertial sensor "STT-IBS" and the sensors attached to an athlete's body.
Figure 7 shows a view of an inertial sensor (left picture) and an athlete wearing these devices in the gymnasium at CAR Venue (right picture); similarly, Figure 17 depicts some optical motion capture markers (left picture) and an athlete using these markers in a training session (right picture).
Figure 8: Optical motion capture markers and an athlete wearing these markers.
The use of motion capture technologies in training sessions for gymnastics and other sports can be an important improvement for the assessment and training of athletes. This technology combined with FMI products and services can provide a significant improvement in training methods and techniques. FMI products allow uploading the motion capture data in cloud making it available to the community. In this way the athlete has the possibility of sharing her/his data with trainers, colleagues and mates who might be geographically distributed, thus enlarge her/his experience in training and gymnastics.
3.4.1.
Overall Goal
The experiment was set up to learn about impact and feasibility of using innovative technology in a sports training environment. The goals of the experiment can be grouped in different categories according to the interests and profiles of the different teams involved in its execution.
27
EXPERIMEDIA For the venue:
The main goal of the 3D Acrobatics experiment for the CAR is to verify that technological assets provided to trainers and athletes are adequate. It will be verified that the infrastructure available at CAR provides full support for the deployment of the experiment and makes it possible to obtain the desired results in terms of improvement of training facilities. The following issues concerning the venue are at the focus of the experiment: Experience on the use of the inertial motion capture system at the CAR Venue. Reliability of the results obtained with the inertial motion capture system. Use and reliability of the connectivity of local network. Practical experiments on the use of GigE cameras available at the training rooms jointly with the inertial motion capture system.
In addition the use of FMI products and services will be verified and the benefits which can be derived from its use in the area of training high performance athletes will be investigated. For trainers and athletes: The main goal of trainers and athletes in gymnastics is to improve the individual technique of the athlete in order to be in the best position to win the competition. For achieving this goal athletes and trainers make use of all the elements which are made available by the training centre where they develop their activities. The following list shows which elements are looked into to improve training and guidance with the help of the experiment: To verify that the inertial motion capture system can be comfortably used by the athlete performing a training session. To verify that the results delivered by the motion capture system provide additional value to the current training methods; trainers and athletes have to corroborate that the data provided by the inertial motion capture system are reliable and useful for improving the athlete's technique and the trainer's approach to training. The athlete's understanding about her/his own motion and technique is significantly increased. The time required to improve a given exercise or technique is decreased if compared with "more traditional" training methods or tools. The cooperative environment provided by the FMI tools significantly increases the training potential for both the trainer and the athlete.
For experimenters: The experimenters are interested in the development of the 3D Acrobatics experiment in order to look into the application of inertial and optical motion capture systems in the training methods of high performance athletes as well as to explore the possibilities of FMI in sports training. From this perspective it can be considered that the goals pursued by the experimenters are a compendium of the particular goals listed above for each different actor in the experiment. The following list summarizes the most relevant goals for the experimenters:
EXPERIMEDIA
Comfort on the use of the motion capture systems in practical training sessions. Reliability of the motion capture data recorded in a practical training session. Experience of the practical use of inertial and optical motion capture systems in the training facilities of a high performance centre. Relevance of motion capture results for the trainer and the athlete for the improvement of the athlete's individual technique. Time required for the application of the motion capture results for the improvement of a practical training session. Practical helpfulness of sharing the motion capture data video and metadata through FMI products in order to improve athlete's individual technique. Investigation on the use of this technology for the improvement of the technique of non-professional athletes. Research on the benefits for non-professional athletes derived from the use of FMI technologies.
3.4.2.
Technical Overview
The technical components or building blocks developed within the 3D Acrobatics experiment are summarized in the following list: 1) Adaptation of the motion capture applications. This task focused on the modification of the software modules that solve the problem of tracking human motion using wireless inertial sensors. Specific biomechanical models for gymnastics have been implemented and used in the experiment runs at CAR Venue. The main modifications aimed at the implementation of protocols for analysis of pommel horse exercises as well as jump analysis. Due to some limitations identified in the use of inertial sensors during the experiment runs it was decided to undertake similar developments in order to use optical motion capture technologies in gymnastic exercises as well as other sports disciplines. 2) Generation of video contents. The objective of this task was the generation of video files from the 3D engine integrated in the capture applications. Originally it was planned to integrate this modification in the motion capture software for inertial sensors. Due to the fact that optical motion capture systems were used as well the same modifications related to video generation were included in the motion capture software with optical technology. The files generated in this step are uploaded to a repository using the SOA implemented by ATOS for the experiment being performed at CAR. The SOA have been used in the implementation of a data manager software. 3) Synchronization of motion capture data with video and metadata. In this task a software module that allows generating metadata with video has been implemented. This software allows the synchronization of the metadata with the video generated by the motion capture software. 4) Visualization of motion capture data for training purposes. The goal of this task is the implementation of a 3D visualization module which runs on the devices hosting the Bluetooth connection with the inertial sensors; the same module has been implemented for the optical motion capture solution. This visualization module allows trainers and
29
EXPERIMEDIA
athletes to visualize athletes motion. It provides instantaneous feedback to the athlete once she/he concludes the execution of her/his exercise. 5) Implementation of VRPN server. This block has been implemented in order to allow synchronization between the motion capture data with the stereoscopic video recorded in the XE6 CONFetti experiment. All the components described in the previous list were used in recording sessions carried out at CAR Venue during the execution of the project. The results of these runs are described in Section 3.4.4.
3.4.3.
Ethics and Privacy
The 3D Acrobatic Sport experiment records actual motion data from real athletes in the CAR Venue. Although the data recorded by the motion capture system do not include any kind of picture or video it can be understood that a given person can be identified by her/his motion characteristics. This situation was discussed during the EXPERIMEDIA General Assembly held in Madrid in January 2013. It was agreed that the current Data Protection Act (DPA) used at CAR was valid for the development of the 3D Acrobatic Sport experiment. After the meeting in Madrid a questionnaire was distributed by the partner responsible for Privacy Impact Assessment in the EXPERIMEDIA project including some questions about the Personal Data Flows regarding the experiments at CAR. This questionnaire was filled and submitted by CAR as well as the experimenters. As a result the partner responsible for PIA proposed an action plan composed of several points. The most relevant suggestion related to 3D Acrobatic Sports experiment was the proposal to sign a controller-processor contract between CAR and STT.
3.4.4.


First experiment run: May 15-16, 2013, Second experiment run: July 24th, 2013 and Third experiment run: September 16-18, 2013
Three different runs of the experiments were performed at CAR Venue in the following dates:
During the experiment runs carried out at CAR Venue the following equipment was used. Inertial motion capture system (see Figure 9) o o o o Set of 6 inertial sensors running at 250 Hz. Bluetooth antenna Tablet Laptop
30
EXPERIMEDIA
Figure 9. Motion capture system with inertial sensors.
Optical motion capture system (see Figure 10) o o o o o Set of 6 infrared cameras with resolution 832x832 pixels running at 250 Hz Set of 6 tripods and 6 camera supports Set of camera cables Ethernet switch PoE Laptop
Figure 10. Motion capture system with inertial sensors.
During the first and second runs of the experiment difficulties were experienced with the use of the inertial motion capture sensors in the execution of some of the exercises proposed for evaluation. These difficulties led to a delay in the reports of this experiment. In order to overcome these difficulties and to achieve the proposal goals it was decided in use not only inertial sensors but also an optical motion capture system. The final run of the experiment was schedule to be carried out simultaneously with other experimenters (PSCN and CERTH). In order to fulfil with the commitments an important development effort was undertaken. The final run of the experiment was performed satisfactorily.
EXPERIMEDIA
The results of this experiment will be reported in D4.7.3, which will be delayed to incorporate the findings of the late September run of the tests and to allow for a thorough analysis and interpretation of the gathered measurements. First experiment run: The first experiment run made use of the protocols implemented for the pommel horse. Previous tests were carried out at the biomechanical laboratory available in CAR premises. The results obtained in these early tests were satisfactory. The run was executed in parallel with CONFetti experiment. The practical execution of the experiment in the gymnasium revealed two different problems with the inertial motion capture system. On one hand the inertial sensors experienced problems to be connected to the host computer using Bluetooth protocol; on the other hand, the relative angles measured at the hips of the athlete showed a lack of accuracy. Further connectivity tests were performed when no additional electronic equipment was present in the gymnasium and no connectivity problems arose. The lack of accuracy was examined in detail. It was concluded that the Kalman filter required more precise data from the triaxial gyroscope. The corrective action proposed was the improvement of the calibration procedure applied to the gyroscopes. This action required the recalibration of the available sensors as well as a change in the firmware of the device. This change in the firmware required in turn a modification in the inertial motion capture software.
Figure 11. Training session in pommel horse; motion capture data and stereoscopic video recorded simultaneously.
Despite the two problems identified during this run of the experiment valid motion capture data were recorded during the training session. This data was recorded simultaneously with the stereoscopic video of the CONFetti experiment. As a result some augmented reality videos were created combining the images recorded by the stereoscopic cameras and the motion capture data streamed by the VRPN server. Figure 11 shows an athlete performing his exercises in the pommel horse and wearing a set of four inertial sensors. The purpose of these sensors was the measurement of the relative angles in the hip joints and lumbar spine.
EXPERIMEDIA Second experiment run:
The second run of the experiment was conducted by STT technicians and CAR technical staff. This experiment was carried out using a new set of inertial sensors which incorporated the improved firmware for gyroscope calibration. All the tests in this run were carried out in the gymnasium available at CAR premises.
Figure 12. Results of jump analysis test (left) and flexion/extension analysis test (right).
Some tests focussed on the measurement of knee flexion motions and jumps. For these purpose two different athletes were appointed. Four inertial sensors were used placed in femur and tibia in both right and left legs simultaneously. Different tests were performed with the two athletes. The results obtained in all the tests were totally accurate and provided the expected feedback. Athletes were satisfied with the results and expressed no objection in the use of the inertial sensors. Figure 12 shows the results obtained in one of these experiments; on the left side a skeleton showing the bones in the legs are depicted; on the right a graph with the flexion/extension angles in the knees. Tests were performed on the same two athletes in the pommel horse. In this case the motion capture data obtained from the inertial sensors was not accurate. As a consequence of these results a detailed revision of the theory, algorithms and implementation of the firmware and software in the inertial sensors was accomplished. Different experiments were designed in order to identify the source of poor accuracy in the motion performed by an athlete in the pommel horse. The results of this research revealed limitations in the implementation of the Kalman filter which computes the orientation of the inertial sensor. The lack of accuracy of the motion capture system with inertial sensors for the pommel horse exercises forced the need of using an optical motion capture system for this type of exercises. Final experiment run: The third run of the 3D Acrobatics experiment made use of an optical motion capture system in order to overcome the limitations identified in the other two runs with the inertial sensors in the pommel horse exercises. During the third run several training sessions of pommel horse exercises were successfully recorded with two different athletes. Figure 13 and Figure 14 show side by side pictures with the athlete performing the exercise and the 3D avatar rendered in the computer screen by the optical motion capture application.
33
EXPERIMEDIA
Figure 13. Training session in the pommel horse: circles and flairs.
Figure 14. Training session in the pommel horse: swings and scissors.
An additional day was devoted to recording training sessions in cycling. The purpose of this experiment was to record simultaneously the motion capture system with the Kinect cycling experiment driven by CERTH in the CAR Venue (see Figure 15). Some of the training sessions in gymnastics and cycling were simultaneously recorded using the stereoscopic HD cameras in the CONFetti experiment.
Figure 15. Training session in cycling. Copyright JOANNEUM RESEARCH and other members of the EXPERIMEDIA consortium 2013 34
EXPERIMEDIA
The collaboration with the CONFetti experiment was materialized by the implementation of a VRPN server as well as the sharing of the 3D model to be used in the combination of the results. This VRPN server was made available for both the inertial and optical motion capture solutions. This collaboration allowed to create augmented reality contents by combining the video recorded in the CONFetti experiment with the 3D models and motion data recorded using the motion capture systems.
3.4.5.
The use of motion capture systems in real training sessions has been very positive. Both trainers and athletes have adopted the technology without any problem. Athletes felt comfortable using the motion capture devices (either inertial sensors or optical markers). The use of those devices did not impose any restriction during the execution of the exercises. The main goals derived from the use of the motion capture systems within the 3D Acrobatics experiment are described in the following points: It has been proven that the motion capture devices (inertial sensors or optical markers) are comfortably used by the athlete performing a training session. It has been verified that the results delivered by the motion capture systems provide additional value to the current training methods; trainers and athletes corroborated that the data provided by the motion capture systems are reliable and useful for improving the athlete's technique and the trainer's approach to training. It has been learnt that an outstanding value provided by the use of motion capture systems is the fact the evaluation is done in an "objective way" thanks to the quantitative values provided by the motion analysis software. It has been confirmed that the use of 3D avatars gives a better understanding of the exercise as well as the perception of the own body and motion; a more detailed analysis can give a better insight about the potential improvements than athletes can made in proprioception by using this kind of technologies. It has been shown that the athlete's understanding about her/his own motion and technique is significantly increased.
The experimenters were interested in the development of the 3D Acrobatics experiment in order to look into the application of inertial motion capture systems in the training methods of high performance athletes as well as to explore the possibilities of FMI in sports training. FMI products and services are seen as a tool to reach a large community of user which are sport fans but not necessarily high performance athletes. The following list summarizes the most relevant lessons learnt by the experimenters: Inertial sensors are practical training sessions. Reliability of the motion capture data recorded in a practical training session has to be improved. Inertial sensors can be used under certain circumstances and for some exercises (for instance jumping or physical evaluation). Inertial as well as optical motion capture systems can be successfully applied in the training facilities of a high performance centre.
35
EXPERIMEDIA
The results provided by motion capture systems are relevant for both the trainer and the athlete for the improvement of the athlete's individual technique. Time required for the application of the motion capture results for the improvement of a practical training session is totally acceptable. It is very helpful the option of sharing the motion capture data, video and metadata through FMI products and services in order to improve athlete's individual technique.
3.4.6.
Dissemination
Dissemination activities carried out by STT are aimed at the promotion of the experiment results among its partners and customers. To this end a logo was designed in order to provide a visual image of the new services and products (see Figure 16).
Figure 16 3D Acrobatics logo
Currently the results obtained in the experiment runs have already been introduced to the following companies and entities: Motion Sports Institute (http://www.motionsportsinstitute.com/SitePages/Home.aspx). Atlantic Health (http://www.atlantichealth.org). Indiana University (http://www.indiana.edu).
3.4.7.
Conclusions
The execution of the 3D Acrobatics experiment led to the conclusion that the proposed technology provides a tremendous added value to the sports community. The combination of motion capture systems with FMI products and services represents an important contribution to the sports community in the area of sports training and performance improvement. The use of motion capture technologies has proven to provide a useful feedback both to athletes and trainers. It gives an instantaneous view of the exercise performed by the athlete offering a better understanding of the training and execution techniques.
36
EXPERIMEDIA
The runs of the experiment also highlighted some limitations on the use of inertial sensors for certain types of evaluation. It has been proven that the inertial motion capture technology can be successfully applied in some exercises. This fact together with the portability of the system (can be used outdoors, can be used in the field, there are no limitations imposed by the environment, etc.) makes this technology an interesting alternative for carrying out analysis in the field. This is an affordable that can be used not only with professional teams but also with amateur teams, schools, etc.
3.5. 3.5.1.
EX8: BLUE Personalized museum experiment Overall Goal
The EXPERIMEDIA BLUE project targeted the enhancement of museum visitors' experience through exploitation of cognitive and visiting styles, individual interests and social gaming. From a user point of view, it aimed at: (1) extending a conventional museum visit, which they can report and share with friends, to the virtual world by offering to Facebook users a dedicated game allowing them to build their own virtual museum with objects they can win inside the game or can be brought back from actual museum visits; (2) provide museum visitors a guided and personalised tour including recommendations and personalised descriptions computed from their cognitive styles and personal interests, both inferred from them playing the game. From a scientific perspective, the BLUE project investigates novel ways to estimate user's cognitive style, visiting style and museum interests through the use of a social networks game in order to provide visitors a personalised guiding enhancing the QoE within the museum. The BLUE project is also studying unique ways to engage the visitor before, during, and after the visit, providing a holistic experience. The BLUE approach to museum personalised visit has been experimented at the Hellenic Cosmos of the Foundation of Hellenic World in Athens, Greece.
3.5.2.
Technical Overview
EXPERIMEDIA-BLUE offers a component bundle called My Personalised Museum Experience, which consists of three main components: (1) My Museum Story (MMS), a social web-based application, which is made for use with Facebook (extracts cognitive profiles, museum interests and visiting styles); (2) My Museum Guide (MMG), a mobile application for Android that constitutes the personalised electronic guide visitors will be given during a museum visit (highly interactive, since the visitor can add comments or pictures); and (3) tailored recommendation engine and personalised exhibition descriptor components that handle recommendations and respectively personalisation for each user. The global architecture is illustrated in 1.
37
EXPERIMEDIA
Figure 17: My Personalised Museum Experience architecture
3.5.3.
Ethics and Privacy
Ethics and privacy has been handled during the BLUE experiment following the Privacy Impact Assessment (PIA) defined in EXPERIMEDIA. Details are given in the EXPERIMEDIA D4.8.1 document. Compliance with the EU directive 95/46/EC has been ensured. Although users were explicitly informed about the nature of the studies and their rights, their consent was also asked and data were also anonymised, confidential and under restricted access. The consent form is provided in electronic format as the first page a user can access when using the MMG application, which she must accept explicitly by checking a box. The text clearly mentions participants' rights as defined by the EU directive 95/46/EC as well as giving them instructions as to how to exercise their rights. It has been validated in its English version by Luxembourgs national agency for data protection and privacy, the CNPD and in its Greek form by the Hellenic Data Protection Authority (HDPA).
3.5.4.
Three different experimentation / analysis phases have been conducted during EXPERIMEDIA BLUE: (1) statistical analysis to correlate MMS game items with players' cognitive style, (2) simulations on personalised museum visit; and (3) experimental runs with visitors guided by MMG. For the first experimentation phase it was found that different game choices correlate with cognitive style dimensions (i.e. choices of avatars, pets, tools, decoration items, fashion items, game type preferences, and specific museum interests). For the second experimentation phase, a crowd simulator was designed aiming at simulating the movements of visitors inside the museum, following or not personalised recommendations. Recommendations effects have been observed for different levels of accuracy in user interests' prediction. We have also examined the effect that such recommendations have on the visiting style of visitors, in order to identify possible approaches, and their context, for further QoE improvements, for the specific problem of museum visit augmentation. During the last experimental phase, data were collected from real users who had played the implemented Facebook game and also visited the using "My Museum Guide" at FHW in four separate sessions. The experiments provided data that can be used to evaluate the experiment both qualitatively and quantitatively. We summarise the results in the following table, in terms of features observed (qualitative study) or hypothesis checked (analytical study) among the four that were defined.
EXPERIMEDIA
Facebook game (MMS): Observed feature / Hypothesis

Gameplay choices
Findings
Hypothesizing that the different choices of avatars, tools and pets would reflect peoples personalities was also supported since 15/20 people explicitly stated that they chose these items since they reflected their personalities. Playing the game, most users did not face any problems, since only 4 out of 30, reported problems. For the first dimension, Extraversion-Introversion, the game correctly estimated this dimension with 69.2%. For the dimension Sensing Intuition the success rate was 58.8%, for the Judger-Perceiver was 77.7%. The lower score was for the ThinkingFeeling dimensions which in only 55%. The Facebook game was able to correctly capture users interests in 91.3% of the times. The available data did not provide support for the first research hypothesis. Users visiting style was correctly predicted from their game choices regarding museum templates only 6 times (27.2%).
Game usability
H1: The Facebook game can reveal players cognitive styles
H2: The Facebook game can predict players museum interests H 4: Facebook game players choices of museum templates can reveal their preferred visiting style for a physical museum
Table 2: results Facebook Game
On-site (with MMG): Observed feature / Hypothesis

QoE: Recommender satisfaction and evaluation
Findings
73% (11 out of 15) of participants followed recommendations. In general 75% (15 /20) of the participants were very positive with the use of the recommender and only a 10% (2/20) mentioned that the recommender should have more information. Although only a 39% (9/23) of the participants answered that they would upload these pictures or that they already had, at the end of the interview when asked about their general views of the experiment, most people said that they really liked the photo feature and most took pictures using the tablets during their visit. All (100%) of the participants explicitly stated that the application was enjoyable and 90% (19/21) of the users mentioned that the information was presented in a satisfying manner. Although all content provided to the users was adaptive and prepared for the different cognitive styles, users did not realize that this content adaptation.
Photo option and online visit sharing
Tablet usability
Effect of personalisation and content adaptation
39
EXPERIMEDIA
H3: The recommender can provide the best-suited exhibitions for each user
The number of stars provided to the users describing the exhibitions relevance to them, was compared to the users self reports on their interests levels for those exhibitions. The average success rate of the recommender was 57.9%. However, when individual categories were studied separately, the recommender was particularly successful to suggest certain types of exhibitions like Ancient Cities (61.3%) and Biology Darwin (65.2%).
Table 3: results on-site experiment run
3.5.5.
In general, Experimedia Blue attempted to combine cultural heritage, social networks, and peoples personalities in a unique way. From the experiments, the following could be retained. Users found the project outcomes (MMS + MMG) highly engaging, capable of improving their Quality of Experience, and the venue (FHW) acknowledged the added-value potential of the project/experiments in improving visitor satisfaction. The analysis of results has shown the potential of using games to deduct people's cognitive profile and interests for museum visits. User satisfaction regarding personalisation aspects indicated an interest for such personalised museum visits even if the exploitation of the cognitive profile for reaching this personalisation remains to be further assessed. The academic community recognised the novelty and the significance of the approach since parts of the present work has been already presented in international conferences and published in conference proceedings. A new road opens in the exploitation of the available information in social networks for adaptivity purposes. The uses of such practices can be numerous and remain to be studied. Moreover, the users found the approach engaging, entertaining and promising. The popularity of social networks and social network games make it an excellent field for use in cultural heritage.
3.5.6.
Dissemination
The project is presented on the web site of Henri Tudor Public Research Centre at http://www.tudor.lu/fr/projets/experimedia-blue. In addition, EXPERIMEDIA-BLUE results have been diffused through presentation in scientific conferences and related published articles (additional conferences articles are targeted as a follow-up of the project and we intend to submit an article to a scientific journal summarising the experimental results): 6) Antoniou, A., Lepouras, G., Lykourentzou, I. and Naudet, Y. 2013. Connecting physical space, human personalities, and social networks: the EXPERIMEDIA Blue project, in Proc. of the International Biennial Conference Hybrid City, Subtle Revolutions. D. Charitos, I. Theona, D. Gragona, H. Rizopoulos, M. Meimaris (Eds). University Research Institute of Applied Communication, Athens, 23-25 May, p. 197-200. 7) Lykourentzou I., Claude X., Naudet Y., Tobias E., Antoniou A., Lepouras G. and Vassilakis C., Improving museum visitors Quality of Experience through intelligent recommendations: A visiting style-based approach, Museums as intelligent environments (MasIE2013)- Workshop co-located with the 9th International Conference on Intelligent Environments - IE'13, 16-19th July 2013, Athens.
EXPERIMEDIA
8) Antoniou, A., Lykourentzou, I., Rompa, J., Tobias, E., Lepouras, G., Vassilakis, C., Naudet, Y. (2013) User Profiling: Can a Facebook Game Reveal Cognitive Style? (poster) Games and Learning Alliance Conference (GALA 2013), 23-25th Oct, 2013, Paris, France. 9) Naudet Y., Lykourentzou I., Tobias, E., Antoniou A., Rompa J. And Lepouras G., Gaming and Cognitive Profiles for Recommendations in Museums, 8th Int. Workshop on Semantic and Social Media Adaptation and Personalization (SMAP2013), 12-13 Dec., 2013, Bayonne, France.
3.5.7.
Conclusions
Our experimental runs revealed that overall, visitors QoE has been enhanced and their final impression within the museum was very good. Roughly (see previous section for exact numbers), 82% liked using the tablet with MMG, 75% had a positive impression using the recommender and 73% actually followed the recommendations. Those results, despite an average precision in recommendations of 58% (on exhibition ratings, on a 1-5 scale) are very encouraging knowing the constraints induced by the museum venue and the few amount of data exploited for profiling users. With the MMS, we were able to show with a high success rate (resp. 91.3% and 67.17%) that user interests and cognitive style could be predicted from a simple game targeting museum topics. Regarding the four hypothesis that we tried to verify, we had the following results. More details about observations and results analysis can be found in EXPERIMEDIA deliverable D4.8.3 Blue Experiment results and evaluation. H1: The Facebook game can reveal players cognitive styles. The data collected in the present study provided partial support for the above hypothesis. The results were more than satisfying and promising for further development. In addition, this is a highly novel approach, not simply because it attempts to estimate users cognitive styles from a simple game, but also because information from social networks can be used in a user profiling process in adaptive systems, whether these will be used in cultural heritage, education, etc. H2: The Facebook game can predict players museum interests. The Facebook game was able to correctly predict players interests and support this Hypothesis. Most users reported that they chose items that reflect their personal interests and not simply because of item aesthetics or image quality. H3: The recommender can provide the best-suited exhibitions for each user. Based on the game data, the recommender was indeed successful in providing the most appropriate exhibitions to the different users but for some and not all exhibitions. The average success in estimating user ratings for each exhibition can be explained by the small data set used for user profiling and exhibition modelling, which makes difficult estimating a rating on a 1-5 scale. The fact however that almost all users were satisfied by recommendations shows that results are encouraging, while leaving a big place for enhancements. H 4: Facebook game players choices of museum templates can reveal their preferred visiting style for a physical museum. The above hypothesis was not supported with the available data. The main reason seems to be the aesthetics of the images used, since certain images seemed to attract the majority of preferences.
EXPERIMEDIA
The EXPERIMEDIA BLUE team faced numerous challenges during the design, implementation and testing of the applications, mainly due to the nature of the venue. FHW was not a traditional museum hosting object-based exhibitions. The layout of the museum and the nature of the exhibitions (i.e. highly targeted, film form, specific time schedule) have affected the outcome of the experiments. Most users were adults invited through Facebook and they found the exhibition content of a school level (some called it childish). This had enormous implications in the present work, since although someone might like Biology for example, it could give the Darwin exhibition a low mark, because it was not of the desired depth. Combined with the fact that most users did not access, read or even notice the adaptive content, possibly adaptivity at this level might not be meaningful. Post-adapting a strongly focused exhibition (as these "exhibitions" were focused on children) does not have an overall effect (positive or negative) on the visitors' experience. Adaptation is something to be considered beforehand, to cater for different target groups and different views/interpretations of the exhibition's message. Altering text messages does not have a strong effect on the experience. However, recommendations are certainly useful. Given that the museum is a small one with a lot of limitations on the visit (hours, presentations, ticketing policy) the overall visitors' feedback is positive. For a large museum, with lots of exhibits (more than what an average visitor would be able to view in a day's visit) and more freedom of movement the impact would have been even greater. In larger museums, such a system would be particularly useful, since it would allow curators to know visitor interests even before their visit, could suggest routes inside the museum to avoid visitor traffic and provide opportunities for both visitors and museums to connect the visit with social networks. In the present work, efficient recommendation depended on two main things: the quantity of information contained in both the user profiles and the items to be recommended; and the matchmaking algorithm that actually computes the items to recommend. In the case of our experimental context, the following issues emerged: (a) Insufficient number of topics attached to each exhibition; (b) Insufficient user profiles: items in the FB game mapped to one topic only and the number of different items was not enough; (c) Number of topics used to characterize both POI (in particular exhibitions) and user interests was clearly not rich enough: a multi-topic classification of exhibition should have been used; (d) Only two rules mapping cognitive styles to exhibitions is not enough, making the recommendations based on cognitive styles not necessarily relevant; (e) There were not enough items to recommend; (f) There were only few participants, but this is not a factor impacting the precision of recommendations. For future research, the following important points will have to be taken into account. To see the effect of knowing the cognitive styles on recommendations, we have to (1) know the users cognitive profile and (2) having a significant set of rules mapping each cognitive style to interests related to exhibitions topics. Then, to be able to increase recommendation accuracy, we need to have: (1) more interests deduced from MMS; (2) exhibitions (resp. POI) described by a set of topics instead of only one. In summary, the profiling process through the MMS game only was too weak to get accurate results. The average results that we have tend to show nevertheless that we should carry on investigating our approach because finally people were satisfied by the recommendations.
42
EXPERIMEDIA
To conclude, users found the BLUE tools (Facebook game - MMS and tablet guide - MMG) highly usable and enjoyable. They also mentioned that it was very interesting to link people personalities and actions prior to the visit, with the physical museum visit. Overall, visitors feedback shows that BLUE tools improve the quality of visitors museum experience, and the interest in individualization of visits and personal technology-enhanced guiding. Moreover, the introduction of social gaming as an extension of visits trough social network has received positive feedback. From a scientific perspective, the interest of gaming to derive visitors interests has been proven, while promising results have been obtained regarding cognitive profile detection through gaming and its use for visit personalization. The novel approach of using a social networks game for museum visitor profiling was successful in numerous ways but most importantly because it opens a road towards the exploitation of the vast quantities of information available in social networks and its use in adaptive technologies.
3.6. 3.6.1.
EX9: REENACT experiment Overall Goal
History is commonly taught in a way that we are only told about who were the belligerent forces, how long the fights lasted for, and who ended up winning. This way, major historical events are put down as occasional events that just happen, that involve two sides (often appearing as the good and the evil forces) and that apparently end fortuitously, as by tossing a coin. Nothing is that simple in reality, and so the common pedagogical approach neglects many facts about the reasons for the battles, alliances and supporters, why things went on the way they did, what were the winning or losing choices, what were the consequences in the short, medium and long terms, etc. As a result, the general awareness of History in society is rather partial and deficient. The goal of the REENACT experiment was to investigate a new approach to improve the understanding of battles and wars that became milestones in human history, taking advantage of the technological features provided by the EXPERIMEDIA facility, and of the spaces, equipment, personnel and contents available to the Foundation of the Hellenic World (FHW). The REENACT proposal is to engage groups of people into an immersive collective experience that tells them about a certain battle or war both from the inside, as reenactors, and from the outside, as historians. Participants learn about the prelude, the course and the aftermath of the event with the aid of multimedia contents and experts. The experiences are organised in three stages: Stage 1 (reenactment) is about involving groups of people and educators/guides in the reenactment of battles. They move around in a room playing the actions defined for a given role by a script of the event. Tactile mobile devices provide the participants with an augmented reality vision resembling a multiplayer role-playing game (RPG). In stage 2 (replay), the participants are taken to a projection room to analyse what has been happening. They have already lived the battle from inside, with a very partial vision, and now it is turn to learn more by watching things from outside, and to see how their recreation compares to the real historic events. The explanations are given by one expert,
43
EXPERIMEDIA
who may be physically present at the projection room or appearing on the screen from a remote location. Finally, in stage 3 (debate), the expert drives a collective brainstorming about the consequences of the conflict in the short, medium and long terms. The projection screen becomes a dynamic big board to display comments posted by the participants, which can be rearranged by the expert as the debate goes on. At any time, the expert can choose multimedia contents to illustrate the different points that are raised.
This general approach has been instantiated in the specific scenario of the Battle of Thermopylae. This was a propitious scenario because the Foundation of the Hellenic World has produced its own multimedia contents about this battle, which is invaluable material for the three aforementioned stages.
3.6.2.
Technical Overview
The REENACT experiences are delivered by four components that rely on/interact with the components of the EXPERIMEDIA facility. The four components are: The reenactors front-end. This is an Android application that delivers the interactions envisaged for the participants during the reenactment, replay and debate stages. It relies on the Pervasive Content Component (PCC) of the EXPERIMEDIA facility to render the augmented reality vision of the reenactments on the participants tactile mobile devices, and on the Social Content Component (SCC) to support messaging, ratings and so on during the replay and debate. The experts front-end. This is a web application providing the functionalities needed by the expert to conduct the replay and debate stages. The experts video flow can be channelled through the Audio-Visual Content Component (AVCC) or through a WebRTC-based implementation created ad hoc by the UVIGO team. The experts text interactions (including the selection of multiple-choice questions) are realised through the SCC as for the other participants. The front-end provides means to manage the arguments raised during the debate, including some features of real-time parsing (e.g. to highlight key words), searching, and filtering of text messages in cases of foul language or disrespectful/offending comments. The experimenters front-end. This is a web application that allows to supervise the operation of the rest of the elements during the REENACT experiences, including manual control over the orchestration of events, means to monitor the reenactors activities (e.g. whether they stay within the reenactors app or temporarily switch to other apps in the mobile devices) and elements to browse the information gathered about QoS, QoE and QoC, which is kept in the Experiment Content Component (ECC). The REENACT server. This is a centralised point for the other components to access pre-recorded contents and streaming flows through the AVCC, to store the records of events raised during the reenactments, to control what is displayed on the different areas of a projection screen during the replay and debate stages, and to upload data to the Experiment Content Component (ECC) for later evaluation. It was also used to enforce the reenactment scripts during the experiment runs from May to August, but this coordination will henceforth be provided by an implementation built by Interactive
44
EXPERIMEDIA
Institute using the Creator environment (bundled with the PCC), which provides a more natural, general and flexible solution to orchestrate events in live multiuser games. The Creator turned out to be a convenient place to lodge text-based and image-based contents linked to the different states of the reenactment game. However, this is not a component to deliver audio or video contents due to potential bandwidth limitations that is up to the AVCC.
3.6.3.
Ethics and Privacy
EXPERIMEDIA enforces strict conditions on the formal assessment of privacy risks to take into account ethical concerns, mainly relating to the protection of personal data and the privacy of the users involved in such experiments. The privacy implications of the REENACT experiment turned out to be very simple due to the following reasons: Participants were identified by nicknames. They took part in a self-contained, one-day session only. There was no processing or profiling of personal data. All the information was gathered through the tablets, not by e-mail or other means. No data was linked to a natural person or disclosed out of the EXPERIMEDIA consortium. Social interactions worked with accounts, events and groups created ad-hoc on Facebook. The expert (the only remote participant) could only see comments linked to nicknames.
These points were explained to the Agencia Espaola de Proteccin de Datos (the Spanish Privacy Commision) in the end of December 2012. Following a telephone conversation, the representative of the agency confirmed that, within the aforementioned guidelines, it was not necessary to do any official paperwork about REENACT.
3.6.4.
The core of the experimentation was done in June 2013 in the Hellenic Cosmos, the venue provided by the FHW in Athens. The data gathered during those sessions was added to the results of ex ante experiments carried out in May in the University of Vigo, aimed at getting early feedback and thereby fix defects in the software or in the experiment design itself. Additional experiments were done during the last week of July and the first week of August in Vigo once again, in order to gather further evidence for the evaluation and to assess any questions that remained unanswered. During the experimentation sessions, the REENACT software was feeding data into the ECC, which will be processed later to evaluate different Quality of Service, Quality of Experience and Quality of Community parameters. On the one hand, the REENACT software kept track of all the movements and actions of the participants during the reenactments, and also of their interactions during the replay and debate stages. Likewise, the application running on the tactile mobile devices provided stats about how and when the participants use its different features and interfaces. On the other hand, the mobile application provided brief questionnaires to gather opinions about the REENACT approach and to rate different features of the experience:
EXPERIMEDIA
educational value, level of entertainment, convenience of the interfaces, quality and completeness of the contents, preferences for certain types of contents, etc. Those ratings are being matched against anonymous information about the participants' educational background and interest in specific topics. Halfway between quantitative and qualitative, the quiz games offered during the replay and debate stages are being used as sources of information about the participants' level of engagement and learning about the historical events. As a research question, it is being checked whether any of the aforementioned parameters depends on the roles played by the participants during the reenactment stage, since it might happen that the QoE measurements are better for someone who has played a main role (say, King Xerxes in the Battle of Thermopylae) than for someone who has played a secondary role (e.g. a Persian infantryman), or maybe that differences appear between winning and losing sides. QoC measurements, again, may reflect both quantitative and qualitative aspects of the community of people that participate in a REENACT session. To this aim, the experimenters are primarily looking at the interactions among the participants during the replay and debate stages, e.g. counting the number of ratings and analyzing the length, mood and depth of the comments they exchange using their tactile mobile devices. Special attention is being paid to what happens among people who did not know each other before, for which they were asked to tick out the nicknames of their acquaintances right before starting the replay stage. Following a preliminary analysis of the data gathered in the experiment runs, it can be said that the users impressions are quite positive in terms of QoE, since participants have provided an average rating of 4.1 out of 5 for the educational potential of the REENACT approach, plus 3.5 for the level of entertainment (which is something that can be improved by devoting more time to the design of reenactment scripts and the content creation part). Furthermore, the participants agreement about REENACT being a valuable asset reached 4.4 out of 5 for History-related museums and 4.1 for educational institutions. QoC measurements suggest that strangers do keep distances during the reenactment, while there is no apparent bias in the ratings given to acquaintances and strangers during the replay and debate stages. QoS measurements have found some excessive latencies in the loading of videos and the augmented reality navigator. The following are some quotes from people relevant to the topics of the REENACT experiment and the EXPERIMEDIA project: I would like to have this if I were a student! someone during the experiments in Athens. A very promising and appealing approach for cultural organizations in the regional scope Cristina Gonzlez, manager of cultural events. The whole approach is very interesting and I'm curious about the results in practice Kleopatra Ferla, historian. A very fresh approach to bring History closer to everyone Beatriz Comendador Rey, historian.
46
EXPERIMEDIA
Just as suitable for primary as for secondary school a member of the Technical Program Committee of SIGMAP 2013, the 10th International Conference on Signal Processing and Multimedia Applications. An exciting and interesting initiative a member of the Technical Program Committee of CSEDU 2013, the 5th International Conference on Computer-Supported Education. Very innovative and well argued contribution! a member of the Technical Program Committee of CSEDU 2013, the 5th International Conference on Computer-Supported Education.
3.6.5.
The results following the evaluation of QoE and QoC data will be used to assess the potential truth of the following claims for the different stakeholders: Museum visitors or students can enjoy new edutainment experiences aimed at improving the understanding of historic events. They can have the opportunity of interacting with one another, and also with geographically distributed experts via user-friendly interfaces. Museum guides and educators can participate in a new type of collective experience, supplementing the expertise and knowledge provided by the experts in replays and debates. Experts can offer their services to collaborate with museum educators in new pedagogical experiences, interacting more closely than ever before with people interested in knowing more about major historical events. They can efficiently browse repositories of multimedia contents to relate historical facts to specific situations raised during the reenactments, and conduct live debates about the potential consequences of the fights in the short, medium and long terms. Content creators/providers and the video game industry can exploit an additional outlet for the multimedia and 3D contents they produce, which can be used to provide historically meaningful explanations to the situations arisen during the reenactments and to the arguments raised during the debates. The contribution of these stakeholders is crucial to make the REENACT experiences visually appealing and catchy.
Commercial exploitation of the REENACT solution could happen through the selling of the technology, its implantation in the venues, training courses for professors, implementation of reenactment scripts and production of multimedia contents.
3.6.6.
Dissemination
During the preparation of the experiment, UVIGO (in collaboration and co-authorship with Prof. Manolis Wallace from the FHW) has made the following submissions to present the ideas and goals behind the REENACT experiment to the scientific community in the area of technology-enhanced learning: 1) The first paper, titled REENACT: Learning about historical battles and wars through augmented reality and role playing An EXPERIMEDIA experiment, was submitted to the 5th International Conference on Computer Supported Education (CSEDU), to be
EXPERIMEDIA
held in Aachen, Germany in May 2013. This paper (6 pages long) was accepted and the oral presentation was delivered by Prof. Yolanda Blanco-Fernndez on May 7th. 2) The second paper, titled REENACT: Augmented reality and collective role playing to enhance the pedagogy of historical events An EXPERIMEDIA Experiment, was first submitted to the 13th IEEE International Conference on Advanced Learning Technologies (ICALT), to be held in Beijing, China in July 2013. The paper was 5 pages long, but due to the absence of results it was accepted as a 2-page paper for poster presentation. UVIGO personnel found it hard to justify the cost of a trip to China for that humble merit, and therefore the submission was withdrawn. The paper was updated with new material and submitted to the 10th International Conference on Signal Processing and Multimedia Applications (SIGMAP), held in Reykjavk, Iceland. The updated paper (8 pages long) was accepted and presented by Prof. Martn Lpez-Nores on July 30th. 3) One extended abstract titled REENACT: Future Media Internet technologies for immersive learning about historical battles and wars about the REENACT experiment was submitted to the eChallenges event, to be held in Dublin, Ireland in October 2013. The submission was accepted and the presentation will be delivered by Prof. Alberto GilSolla on October 10th. 4) Another paper, titled Leveraging short-lived social networks in museums to engage people in History teaching has been submitted to the 8th International Workshop on Semantic and Social Media Adaptation and Personalization (SMAP), which is to be help in Bayonne, France in December 2013. The notification of acceptance or rejection should be received before the end of August. In addition to submissions to conferences, the REENACT experiment has made it into local and regional media, as it can be seen in the following links: http://duvi.uvigo.es/index.php?option=com_content&task=view&id=6763&Itemid=23 http://codigocero.com/O-grupo-SSI-da-Uvigo-crea-un http://www.farodevigo.es/gran-vigo/2012/12/19/universidad-recluta-300termopilas/729080.html http://atlanttic.uvigo.es/index.php?option=com_content&view=article&id=199:viviren-tu-piel-una-batalla-historica-mientras-visitas-unmuseo&catid=37:noticias&Itemid=0&lang=en
Finally, Prof. Martn Lpez-Nores was invited to give a talk about REENACT at the event A historia no medio (History in the media, http://historiaourense.blogspot.com.es/p/ahistoria-no-medio.html) on April 5th, 2013. This event was a space to discuss the connection of History with other disciplines, making visible the role of the historian in the world and the social setting of the speech as a collective construction, structured in the form of short presentations and panel discussions. UVIGO took advantage of this event as an opportunity to get additional feedback about the design of the REENACT experiment and also to recruit volunteers to participate in local experimentation sessions.
48
EXPERIMEDIA
3.6.7.
Lessons learnt for FMI/FIRE
Notwithstanding the fact that the analysis of the data gathered during the experiment runs is yet to be completed, it is already possible to draw some conclusions from REENACT in terms of lessons learnt for FMI/FIRE. The major point for the UVIGO team is that a technological facility like the one that is being built within EXPERIMEDIA can really reduce the effort needed to deploy FMI applications to enrich peoples access to information that is relevant to their current physical and social context. The specifics of REENACT were particularly relevant to the development of the Pervasive Content Component (PCC), which in the end provided suitable solutions to deal with the augmented reality features of the proposal and with the very complex problem of ensuring proper coordination among the actions of different players during the reenactment stage. The AVCC and the SCC are undoubtedly useful for FMI experiments in general, while the ECC is seen by the UVIGO team as a major asset that brings coherence to the whole thing and eases a lot of work. The evolution of the EXPERIMEDIA facility during the last year was very positive, to the point that it would take much less effort to create the REENACT system from scratch now (August 2013) than at the start of the experiment (October 2012). New features and, above all, new and better documentation produced during these months will surely be helpful for new experimenters to focus more on the added value of their ideas and the corresponding applications than on the technical issues that must be addressed to make them run properly. The UVIGO team would like to highlight the importance of documentation provided by Interactive Institute about different ways to specify interactions in live multiuser games, since it provides valuable insight into how to do things in a way that is easy to understand, maintain and scale. Similar pieces of documentation about ways to design and implement AR features properly could be an interesting contribution, too.
3.6.8.
Conclusions
As a final word, UVIGO can say that the experience of building and testing the REENACT approach on top of the EXPERIMEDIA facility has been positive and useful, and at the same time they believe that their work has served to push the facility building tasks forward to a reasonable extent. The results obtained thus far are positive and encouraging to keep exploring the possibilities to have REENACT deployed or exploited commercially in History-related museums and/or educational institutions.
49
EXPERIMEDIA
4. Summary and conclusion

The six experiments from the first open call of EXPERIMEDIA were generally completed successfully, though it has to be said that the evolution of the facility with the help of the requirements and the feedback of the experiments was such that in most cases the final runs had to be delayed to be done with better suited software. A lot of work in further developing the baseline components as well as in integrating various elements has been done and this made the conclusion of the experiments within the given timeframe possible, even though some difficulties had to be overcome. The results gave valuable insights into the possibilities offered by FMI technologies and were an important indicator of the feasibility of such integrations. With the lessons learnt EXPERIMEDIA now starts another round of experiments which will hopefully benefit from the evolution in the facility and bring further insights into this field.
50

D1.2.3 Initial Experiment Review Report v1.0 PDF

Încărcat de

Informații document

Titlu original

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

D1.2.3 Initial Experiment Review Report v1.0 PDF

Încărcat de

Drepturi de autor:

Formate disponibile

D1.2.

3 Initial Experiment Review Report

3. Overview of first Open Call experiments

Venue Schladming Schladming CAR CAR FHW FHW

In the following chapters the experiments are explained in detail.

EX4: Digital Schladming Overall Goal

Figure 1: Schladming Pinboard Architecture

Figure 3 - The MySchladming interface

Ethics and Privacy

Experiment Runs and Results

Value Impact Assessment

Figure 4 - Digital Schladming logo

Figure 5. The mobile Augmented reality application.

The LIME player: https://github.com/tkurz/lime

Figure 6. The interactive video player.

Ethics and Privacy

Experiment Runs and Results

Value Impact Assessment

Ethics and Privacy

Experiment Runs and Results

Value Impact Assessment

EX7: 3D Acrobatic Sports

EXPERIMEDIA For the venue:

Ethics and Privacy

Experiment Runs and Results

Figure 9. Motion capture system with inertial sensors.

Figure 10. Motion capture system with inertial sensors.

EXPERIMEDIA Second experiment run:

Value Impact Assessment

Figure 16 3D Acrobatics logo

EX8: BLUE Personalized museum experiment Overall Goal

Figure 17: My Personalised Museum Experience architecture

Ethics and Privacy

Experiment Runs and Results

Facebook game (MMS): Observed feature / Hypothesis

H1: The Facebook game can reveal players cognitive styles

Table 2: results Facebook Game

On-site (with MMG): Observed feature / Hypothesis

Photo option and online visit sharing

Effect of personalisation and content adaptation

Table 3: results on-site experiment run

Value Impact Assessment

EX9: REENACT experiment Overall Goal

Ethics and Privacy

Experiment Runs and Results

Value Impact Assessment

Lessons learnt for FMI/FIRE

4. Summary and conclusion

S-ar putea să vă placă și