D4.11.

1 ICaCoT Problem Statement and Requirements

2014-01-17

Omar Niamut; Ray van Brandenburg (TNO)

This deliverable provides a detailed overview about the iCaCoT experiment problem statement and requirements for the Schladming Venue. This document provides EXPERIMEDIA facility operators and software components developers with requirements related to physical and digital assets and infrastructure for the implementation and execution of the iCaCoT experiment. It also provides planning information and a description of the measurement methodology.

www.experimedia.eu

EXPERIMEDIA
Project acronym EXPERIMEDIA

Dissemination level: PU

Full title Experiments in live social and networked media experiences Grant agreement number 287966 Funding scheme Large-scale Integrating Project (IP) Work programme topic Objective ICT-2011.1.6 Future Internet Research and Experimentation (FIRE) Project start date 2011-10-01 Project duration 36 months Activity 4 Experimentation Workpackage 4.11 EX11: interactive Camera-based Coaching and Training (iCaCoT) Deliverable lead organisation TNO Authors Omar Niamut; Ray van Brandenburg; Bastiaan Wissingh (TNO) Reviewers Gert Kienast (Joanneum DIGITAL) Version 1.0 Status Final Dissemination level PU: Public Due date PM27 (2013-12-31) Delivery date 2014-01-17

Copyright TNO and other members of the EXPERIMEDIA consortium 2013-2014

EXPERIMEDIA

Dissemination level: PU

Table of Contents
1. 2. 3. Executive Summary............................................................................................................................ 4 Introduction ........................................................................................................................................ 5 2.1. 3.1. 3.2. 3.3. 3.4. Concept ...................................................................................................................................... 6 Learning Objectives .................................................................................................................. 8 Usage Scenarios ......................................................................................................................... 9 Participants and Venue........................................................................................................... 10 Experiment Procedure ........................................................................................................... 12 Experiment Description .................................................................................................................... 8

3.4.1. Preparation ........................................................................................................................... 12 3.4.2. Execution ............................................................................................................................. 13 3.4.3. Analysis ................................................................................................................................. 13 3.5. 3.6. Background .............................................................................................................................. 14 Assumptions and Preconditions ........................................................................................... 14

3.6.1. Use of EXPERIMEDIA Baseline Components ............................................................ 15 3.6.2. Use of Schladming venue facilities ................................................................................... 17 4. Ethics and Privacy ............................................................................................................................ 18 4.1. 4.2. 5. 5.1. 5.2. 5.3. 6. 7. General thoughts and ideas ................................................................................................... 18 Feedback from EAB and DPB meetings ............................................................................ 19 Requirements ........................................................................................................................... 22 System Architecture ................................................................................................................ 24 Content Lifecycle .................................................................................................................... 25

Experiment Design .......................................................................................................................... 22

Plan for Implementation ................................................................................................................. 26 Conclusion ......................................................................................................................................... 27

Copyright TNO and other members of the EXPERIMEDIA consortium 2013-2014

EXPERIMEDIA

Dissemination level: PU

1. Executive Summary
This deliverable reports on the design of the EXPERIMEDIA experiment for interactive camerabased coaching and training (iCaCoT), which focuses on interactive video navigation for camerabased coaching and training for visitors and sports enthusiasts in Schladming. The goal of this experiment is to showcase the unique properties of tiled adaptive streaming, a technology that was developed in the EU FP7 project FascinatE and that allows users to navigate freely through high resolution video panoramas, while the application reduces bandwidth requirements by only sending that part of the video a user is interested in. During this experiment, we will develop the iCaCoT application and run two experimental periods at the EXPERIMEDIA Schladming venue to test the feasibility of the concept of interactive camera-based coaching and training. Through the experiment, the existing EXPERIMEDIA technical components, notably the Experiment Content Component, will be used and tested. The experiment will have impact by raising the attractiveness of Schladming as a venue for ICT-driven sport tourism, by providing valuable input to MPEG DASH standardization, by developing technology for large-scale tests by the BBC beyond Schladming and EXPERIMEDIA, and by enhancing the potential for exploitation of tiled streaming technology.

Copyright TNO and other members of the EXPERIMEDIA consortium 2013-2014

EXPERIMEDIA

Dissemination level: PU

2. Introduction
In the Future Media Internet (FMI), adaptive streaming technologies such as MPEG Dynamic Adaptive Streaming over HTTP (MPEG-DASH1) and HTTP Live Streaming (HLS2) are expected to represent a majority share of all video streaming towards mobile devices. With the increasing importance of over-the-top video and connected devices, such as Smart TVs, tablets and networked media players, adaptive streaming is rapidly becoming the most used streaming technology used over the internet, replacing traditional streaming technologies such as RTP and RTMP in all areas apart from managed video delivery networks. Up until now, adaptive streaming is mostly used as a straightforward over-the-top method to increase perceived Quality of Service (QoS) by dividing content in multiple chunks and making each chunk available in multiple qualities, or bitrates, and having a client seamlessly switch between these qualities when the network performance forces it do so. We believe that the unique properties of adaptive streaming allow it to be used for much more, and envision entirely new use cases and personalized media services becoming available through the use of segmented media files. One specific use case is centred around interactive video navigation, and the underlying technology to support this use case is referred to as tiled adaptive streaming3. With the advent of high resolution and panoramic cameras, which are able to record in HD or higher resolutions, it becomes interesting to also segment content spatially. By dividing a video frame up into multiple tiles, where each tile contains a particular area of the video, a client can choose to only receive certain areas of a video. Such a tiled streaming solution enables an inherently scalable method for users to interact with and navigate in a video using pan-tilt-zoom (PTZ) commands. In the EU FP7 project FascinatE4 we have developed a tiled streaming application, available for both iPhones and iPads, that provides PTZ video using tiled adaptive streaming (see Annex I). Using touch gestures on smartphones and tablets, users are able to navigate freely through high resolution video panoramas, while the application limits bandwidth requirements by only sending that part of the video a user is interested in. The tiled streaming application, as developed within FascinatE, is based on regular HTTP adaptive streaming approaches such as HLS and MPEG-DASH. Thus, it is completely client-based and therefore inherently scalable; every individual tile segment can be independently cached in the network. See Figure 1 for a visual introduction of the tiled streaming application. For an online demonstration video, the reader is referred to http://vimeo.com/67199886.

T. Stockhammer, Dynamic Adaptive Streaming over HTTP - Standards and Design Principles, MMSys11, February 2325, 2011, San Jose, California, USA. 2http://developer.apple.com/library/mac/#documentation/NetworkingInternet/Conceptual/StreamingMediaGui de/Introduction/Introduction.html 3 O.A. Niamut, M.J. Prins, R. van Brandenburg, A. Havekes Spatial Tiling And Streaming In An Immersive Media Delivery Network, in Adjunct Proceedings of EuroITV 2011, Lisbon, Portugal, June 2011. 4 http://www.fascinate-project.eu/
1

Copyright TNO and other members of the EXPERIMEDIA consortium 2013-2014

EXPERIMEDIA

Dissemination level: PU

2.1.

Concept

Figure 1: Impression of interaction with video on a tablet device.

Through many discussions with interested parties at our technology demonstrations, we noted that the concept of tiled adaptive streaming is particularly well suited to training and coaching applications. That is, using a smartphone or tablet, a coach is able to zoom in on his trainee coming down the mountain, focusing on specific areas, both temporally as well as spatially. We refer to this as interactive camera-based coaching and training. This is especially useful in snow activities, where the exact line followed by the trainee is not known beforehand and can therefore only be captured using a wide-angle lens located relatively far from the action. See Figure 2 for an example of such navigation. By pausing the video at key moments, trainer and trainee can focus on and discuss details of the performance. By placing multiple high resolution cameras around strategic positions, it is even possible for a trainer to view a moment from different angles. The tiled streaming application facilitates this using high-accuracy synchronization techniques, making sure all cameras are synchronized frame-accurately. With the interactive Camera-based Coaching and Training (iCaCoT) experiment, we want to test how actual users use the concept of tiled adaptive streaming in a live and real-world environment. We envision that several statically mounted high resolution (i.e. HD or higher) cameras will be placed at a suitable location around the Schladming venue, such as around a ski slope for training or a fun park for winter sport enthusiasts. The video recorded by these cameras will be spatially segmented in real-time using tiled streaming ingest components. Once the video has been tiled, it will be offered to users in and around Schladming venue who can use their own smartphone and tablet to download a Schladming-specific version of our tiled streaming application from the Apple App store and navigate around the video, seeing themselves coming down the slope or performing tricks in the fun park.

Copyright TNO and other members of the EXPERIMEDIA consortium 2013-2014

EXPERIMEDIA

Dissemination level: PU

Figure 2: Interactive camera-based coaching using tiled adaptive streaming for video navigation.

Copyright TNO and other members of the EXPERIMEDIA consortium 2013-2014

EXPERIMEDIA

Dissemination level: PU

3. Experiment Description
The proposed experiment is aimed at testing tiled adaptive streaming technology in practice, specifically as a tool for training and coaching and as a novel way for Schladming visitors to record themselves coming down the mountain and sharing their experiences with friends and family. The concept of tiled adaptive streaming has already been developed into a mobile application and has been tested in a full live setting5. The experiment will focus on obtaining user feedback and large-scale testing in an outdoor environment. The Schladming venue provides the required level of scale for this experiment, and it can provide its visitors with a novel experience not found at any other ski resort. The experiment has the following three main objectives: 1) Capture the user experience when interacting with the tiled streaming application; 2) Test the feasibility of tiled adaptive streaming as a tool for training and coaching activities; 3) Leveraging information obtained through large-scale user tests to improve tiled adaptive streaming bandwidth efficiency.

3.1.

Learning Objectives

Objective 01: Capture the user experience when interacting with the tiled streaming application Obtaining user feedback on the concept of tiled adaptive streaming in general, and specifically on the quality of service as delivered by our current implementation, is our most important objective for this experiment. Instead of performing controlled lab-trials with users, EXPERIMEDIA allows users in real-world environments to test the novel methods of user interaction that are possible when allowing pan-tile-zoom interaction with video content and navigating between different camera sources. We are interested in evaluating the user experience from a Quality of Service (QoS) and a Quality of Experience (QoE) perspective, to allow for further improvement of both the application and the underlying streaming technology. An element that is of special interest during the experiments is the effect of tile-switching delay on the Quality of Experience perceived by a navigating user. Since there is a trade-off between bandwidth efficiency and this delay, obtaining feedback on what users perceive as an acceptable delay is vital information. Related to this is the quality of this video and the bit rate at which the tiles will need to be encoded. The experiments will also collect feedback on the intuitiveness of use of the application, the user interface and the methods for navigating between different camera sources. Furthermore, we aim to investigate how we can best allow users to locate themselves temporally within recorded video footage once they have arrived safely at the bottom of the mountain. Main success criteria for this objective are both quantity of quality of collected user feedback. While the automatic logging of all user navigation within the app will provide a wealth of information in how users use the concept of Tiled Streaming to navigate in the video, in-app
5

http://manchestergazette.co.uk/salford-university-brings-the-future-of-live-tv-to-mediacityuk/ 8

Copyright TNO and other members of the EXPERIMEDIA consortium 2013-2014

EXPERIMEDIA

Dissemination level: PU

questionnaires will be used to provide qualitative information on user perception and Quality of Experience. To assure the quality of this information, it is vital that these questionnaires be unobtrusive and quick. Objective 02: Test the feasibility of tiled adaptive streaming as a tool for training and coaching activities Another important objective for this experiment is assessing the potential of tiled adaptive streaming as a tool for training and coaching applications. Whereas we have previously focused on the entertainment and security industries as primary domains for the tiled streaming application, the combination of spatial and temporal navigation make tiled streaming particularly well suited for training and coaching. In particular in outdoors environments where a wide viewing area needs to be covered while the filmed subject itself is usually small, interactive video navigation can enhance the possibilities for (self-)training and coaching. For coaching and training purposes, the location of the various cameras is very important. Depending on the particular activity being trained for, one can imagine different camera setups being most useful, from multiple sequential cameras being positioned in a line on the side of ski slope, to different cameras being arranged in a circle to film the same location, e.g. a jump in a fun park, from different angles. During the experiment at least two of such different arrangements will be tested in consultation with local ski professionals. In this test, we will investigate how trainers use the system, and whether the tiled streaming concept helps them in their instruction activities. We will track usage as well as present trainers with questionnaires. These questionnaires will be built into the app for direct feedback. Objective 03: Leveraging information obtained through large-scale user tests to improve tiled adaptive streaming bandwidth efficiency We will obtain user logs to enhance the technical aspects of tiled adaptive streaming in a given camera setup. By mining user logs it is possible to determine areas in the recorded video that are of particular interest to users, creating heat-maps of user activity. Such heat-maps can subsequently be used to optimize the video delivery aspects of tiled streaming, such as optimizing the tile grid to fit these areas, thereby reducing the number of tiles required to reconstruct the areas, and with it the necessary bandwidth and tile-switching delay. They can also be used to (automatically) enhance the user interface, by offering direct shortcuts to popular viewpoints. We will test the hypothesis that certain areas in a video are far more popular than average. In subsequent iterations, different algorithms for determining tile layout will be tested by actual users, while comparing average bandwidth and tile-switching delay.

3.2.

Usage Scenarios

Two scenarios have been discussed with the Schladming venue. In order maximize both Schladming visitors' experiences while visiting the region and business values for local stakeholders without sacrificing the research questions, we aim to cater for both scenarios within the overall experiment lifetime.
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-2014 9

EXPERIMEDIA

Dissemination level: PU

One scenario focusses on young ski athletes, training for their participation in large and national events. Lukas Seyfried from Schladming20306, possibly with one or more of his coaching colleagues, will participate in this experiment, along with his students, to use the app in his training approach and to provide direct feedback to students. A second scenario focusses on visitors of a Schladming funpark, interested in a mix of training and entertainment. Important features here are the possibilities of social sharing and ease-of-use.

3.3.

Participants and Venue

The experiment is situated in the Schladming region. The experiment will consist of two rounds (see section REF) at two locations. The Reiteralm area seems to be especially well-suited for (semi-)pro coaching and training purposes, while the funpark in Planai is better suited for experiments involving tourists. The first experimentation period of 3-4 days will take place in the week of 17-21 February. This first round of experiments will take place on Reiteralm, either on the Grande Finale slope, or around the Muldenlift slope. See Figure 3 below.

Figure 3: Reiteralm locations for 1st experimental run.

6

http://www.schladming2030.at/ 10

Copyright TNO and other members of the EXPERIMEDIA consortium 2013-2014

EXPERIMEDIA

Dissemination level: PU

The second experimentation period of 7-8 days will take place in the 3rd or 4th week of March. The second round of experiments will be split in two halves. The first half will focus again on training and coaching in the Reiteralm area. For the second half, the system will be re-located to the Planai funpark, where the experiment will be targeted at tourists using the funpark. System setup is expected to take one day, whereas system teardown will be 2-4 hours.

Figure 4: Planai funpark.

Figure 5: Impressions form Schladming venue visit. Reiteralm (left) and Planai (right).

Given the double focus of the experiment, the involved actors are as follows:

Copyright TNO and other members of the EXPERIMEDIA consortium 2013-2014

11

EXPERIMEDIA End-users of the mobile application

Dissemination level: PU

Persons using the iCaCoT application during the two experimental periods are providing the majority of QoS and QoE data. We distinguish between three groups of end-users: 1) Trainers/coaches (2-5 people): Are persons who will use initial versions of the application in the first experiment run and will provide feedback for adaptation of the experiment and application. 2) Students (2-5 people): Are persons who will participate in the first experiment run. For both groups 1) and 2), consent is obtained a priori. 3) Volunteering visitors of Schladming (5-20). Are funpark visitor who volunteer to gather rich QoE feedback. Specifically, they will install and use the application during their visit to the funpark and participate in post-hoc interviews. Consent to participate in the experiment will be asked within the application for this group and QoE measures will be gathered through in-app feedback. This is the largest group of users contributing to QoS and QoE data. The recruiting will be done through our venue partner, an open call (poster, flyer, social media, ) and with help from Evolaris through their Living Lab, in which they have a database of hundreds of people potentially being interested in taking part in studies. All participants will have to sign a written consent form. The participants will get a day pass for the ski resort as incentive or a voucher for a meal and drinks. Local stakeholders Local stakeholders in Schladming, primarily representatives of the cable car companies who are in charge of the ski areas and local installations. They will support the installation and maintenance of the camera setups. For example, to install and maintain the video cameras we need some support from the cable car operators. This is being organized by our venue partner Schladming 2030. Experimenter Experimenters will overview the execution and analysis of the experiment. Participants in the first run will be attracted through contacts within Schladming 2030. Existing groups of coaches and students will be involved and will participate in experiment preparations. Participants in the second run will be attracted through contacts within Schladming 2030 and through joint efforts with Evolaris, e.g. through the Evolaris Living Lab in Schladming.

3.4. 3.4.1.

Experiment Procedure Preparation

As a preparation for the experiment, we set the requirements and perform interface/integration development. Specifically, we define the details of the experiment that will be carried out, aligned
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-2014 12

EXPERIMEDIA

Dissemination level: PU

to the requirements from all stakeholders involved, i.e. the experimenters, the venue partner and the other EXPERIMEDIA partners. Finally, the integration of and interfacing between iCaCoT and EXPERIMEDIA components will be performed. Based on the EXPERIMEDIA documentation and discussion with project partners, we identify the exact interfaces to the EXPERIMEDIA technical components, notably the Experiment Content Component (ECC), and in later stages the Audio Visual Content Component (AVCC) and Social Content Component (SCC). The integration between these components and the tiled adaptive streaming components that were developed earlier will take place during the development phase. As such, the results of the preparation phase are mainly documented in this deliverable, i.e. the full experimental plan, as well as the functional description of the developed components and test methodology.

3.4.2.

Execution

During the execution phase of the experiment, we run two experimentation periods in two locations of the Schladming venue. The Reiteralm area seems to be especially well-suited for (semi-)pro coaching and training purposes, while the funpark in Planai is better suited for experiments involving tourists. In the first experimental period, a short (3-4 days) round, we will have a focus on technical testing. For the first round of experiments, high-quality feedback on the use of the application and ways to improve it is most relevant. In these first experiments Lukas Seyfried from Schladming2030, possibly with one or more of his coaching colleagues, will participate in this experiment, along with his students, to use the app in his training approach and to provide direct feedback to students. Important elements here are trick-play (pause, replay, slow-motion, etc.) and drawing tools. Feedback will be used to improve the application during the experimentation period. These tests will most likely lead to further development and modification to the iCaCoT application and the tiled adaptive streaming technology. The results of this task will be documented in deliverable D4.11.2, which will provide the technical details regarding the tested components and a progress report including the preliminary results of the first experiment. This first experimentation period will be the week of 17-21 February. The second series of experiments, a longer (7-8 days) second round, will be split in two halves. The first half will focus again on training and coaching in the Reiteralm area. For the second half, the system will be re-located to the Planai funpark, where the experiment and application will be targeted at tourists using the funpark. Important features here are the possibilities of social sharing and ease-of-use. The start for this round will be the 3rd week of March.

3.4.3.

Analysis

In the third and last phase of the experiment, we evaluate and collate the results obtained in both experimentation periods. The results of the first experimentation period will be analysed, both from a technical perspective as well from the received user feedback, and will feed to further development of components. The results of the second experimentation period will be mainly used for evaluating the overall user satisfaction, as well as for dissemination purposes. The results of this task will be documented in D4.11.3, which will include the final experiment results and evaluation, as well as a progress report on (planned) dissemination activities. A showcase will be prepared based on the experiment outcomes for dissemination purposes.
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-2014 13

EXPERIMEDIA Figure 6 below depicts the overall planning of the various phases.
Dec-2013 Nov-2013

Dissemination level: PU

Feb-2014

May-2014

Oct-2013

Jan-2014

Mar-2014

Apr-2014

Activity T4.x.1: Requirements and Development T4.x.2: Planning and Experimentation Experiments T4.x.3: Evaluation and Dissemination

Finish date Month 4 Month 6 Month 8

D4.x.1 D4.x.2 E1 E2 D4.x.3

Figure 6: overall planning of the various phases.

3.5.

Background

We will commit our tiled streaming testbed and application as background knowledge and technology for usage and development in this experiment. This testbed offers a mechanism for scalable delivery of high resolution video to a variety of devices and allows for interaction and navigation, such as pan/tilt/zoom. We own IPR on the underlying tiled adaptive streaming technology. See http://vimeo.com/67199886 for a demonstration video. Key features of the testbed include: Only streaming that part of a video the user is interested in, making optimal use of available bandwidth. Intelligence in the client, allowing for scalability to millions of devices without any serverside processing. Fully HTTP-based, leveraging existing Content Delivery Network (CDN) infrastructure. Aligned with current adaptive streaming solutions, such as Apple HLS and MPEGDASH All decoding performed using hardware decoder, saving battery life on the mobile device. Layered approach allows for easy integration in existing mobile applications

Furthermore, in order to carry out the proposed experiment, several high-resolution video capture and streaming nodes will have to be acquired and installed. Such a node will consist of relatively low-cost HD video cameras, i.e. the GoPro Hero 3+ camera7 and a transmission node to transport the video stream to the tiled streaming ingest node. The camera streams will be transported via a wired link to the tiled streaming ingest.

3.6.

Assumptions and Preconditions

We make several assumptions on the features offered by the EXPERIMEDIA Baseline components and on the facilities and support offered by the Schladming venue. We assume the participants are regularly skiing on slopes (at least once a year), have an iOS device (e.g. iPhone or iPad) and are willing to have a dedicated application installed. To make sure, these assumptions are being met, we will select the participants based on these parameters.

http://gopro.com/hd-hero3-cameras 14

Copyright TNO and other members of the EXPERIMEDIA consortium 2013-2014

EXPERIMEDIA

Dissemination level: PU

Furthermore, we assume there is good local Wi-Fi coverage at the locations, and preferably also reasonable outside Internet connection for social and sharing functionality.

3.6.1.

Use of EXPERIMEDIA Baseline Components

We plan to make extensive use of several key EXPERIMEDIA baseline components, in the following ways: Experiment Content Component As indicated in Objectives 01 and 02, an important element of the experiment will be to log all commands carried out by users within the tiled streaming application and aggregate them using the EXPERIMEDIA Experiment Content Component (ECC). Such logs, in anonymized form, are interesting for a number of reasons. First and foremost, as a valuable input to user experience research; getting feedback from users to see how they use the concept of tiled adaptive streaming in practice. This kind of information is invaluable for further improving the application and bringing it to market. Apart from recording the raw navigation commands, the application will also be extended with more direct ways of getting user feedback, such as periodic questionnaires actively asking users for feedback on how they experience different aspects of the application. For the EXPERIMEDIA project, the logging of all navigation commands generated by the tiled streaming application will present an interesting case to test the scalability of the ECC. Another advantage of collecting navigation commands from users is that it allows us to further optimize the bandwidth efficiency of tiled adaptive streaming, and thereby increase the QoE for the user. Based on user input, heat-maps of areas in the video panorama which are of above average interest to users (i.e. to which users navigate often) can be created. Examples of such areas could for example be ski-jumps in a fun park or busy areas on the ski piste, as depicted in Figure 7.

Figure 7: Hotspots identified through a heat-map analysis.

Copyright TNO and other members of the EXPERIMEDIA consortium 2013-2014

15

EXPERIMEDIA

Dissemination level: PU

Using these heat-maps, derived from user interactions, it is then possible to optimize the delivery efficiency of tiled streaming even further8. As an example, areas which are frequently requested could be tiled using a finer granularity grid, reducing wasted pixels (pixels that fall outside the requested region-of-interest but which have to be requested since they are part of a required tile). Such optimizations can result in more efficient bandwidth usage as well as reducing switching times for the user while navigating through the video, increasing QoE. Additionally, the hotspots indicated in the heat-maps can be made directly accessible for fast navigation. Social Content Component Using the EXPERIMEDIA Social Content Component (SCC), which will be integrated in the tiled streaming application, it will be possible for users to share their unique path through the video with friends, family or training partners. Instead of creating individual video encodings for every user-generated video, with tiled adaptive streaming it is only necessary to record the coordinates in the video as the user navigates through the panorama, and share these scripts with others. Using embedded timestamps it is possible for everyone with access to the tiled panorama to recreate the navigation path on their own device. Alternatively, the recorded coordinates can be used by a cloud-based service to generate a YouTube video and post it online. Audio-visual Content Component We aim to use the AVCC as a streaming server for hosting and delivering tiled streams. An interesting side-experiment that can be carried out using the obtained heat-maps is to publish a (live) stream of a user navigating through the panorama and view it on public screens in and around the Schladming venue, such as in the Schladming Media Centre. These could be the most popular streams, as derived from the heat-maps. Using the EXPERIMEDIA Audio Visual Content Component (AVCC), a stream showing these areas can be published and distributed to various systems and devices. Note that the combined usage of the AVCC and SCC has previously been investigated and developed in the Social Annotation service. An adapted version of this combination seems suited to our purpose. Figure 8 depicts the foreseen interworking between EXPERIMEDA baseline components and components of our tiled streaming testbed and application in the iCaCoT experiment.

Crowdsourced and Content-based Zoomable Video Player, Carlier et al, Proceedings of the 19th ACM International Conference on Multimedia (MM'11), Scottsdale, AZ, 28 November - 1 December 2011.
8

Copyright TNO and other members of the EXPERIMEDIA consortium 2013-2014

16

EXPERIMEDIA

Dissemination level: PU

Figure 8: Connection between EXPERIMEDIA components and iCaCoT components.

3.6.2.

Use of Schladming venue facilities

Help in choosing suitable location for installing camera(s); Logistical support in installing and maintenance of camera(s) and cabling during experiments; Wi-Fi connection between the segment server and user clients; Support in creating awareness for experiment.

We expect support from the venue regarding the following points:

Copyright TNO and other members of the EXPERIMEDIA consortium 2013-2014

17

EXPERIMEDIA

Dissemination level: PU

4. Ethics and Privacy

4.1.

General thoughts and ideas

Transparency through notification screen to explain nature and purpose of experiment; Obtain explicit informed consent through 'I agree' button in the application; Observational research in situations where those observed would expect to be observed; Collected data will only be used for purposes of experiment and shared with participants.

In general, we aim to adhere to the following principles with respect to ethics and privacy:

In more detail: Informed Consent Any users of the iCaCoT application are subject to informed consent. Upon starting the mobile application, a notification screen will explain the nature and purpose of the experiment, the experiments goals and the nature and handling of the gathered data. A user can then able to agree to these aspects of the experiment by checking a box. Without given consent, a user will not be able to use the iCaCoT application. Involving minors - For the first experimental run, and the first half of the second experimental run, our experiment involves minors; i.e. students of Schladming coaches and trainers. We will ensure that consent is asked from their parents and will arrange for the support of their trainers and coaches to monitor the process. The students do not need to download the application. Observational Research - Since part of the proposed experiment is expected to include observational research through the use of statically mounted video cameras, it will respect the privacy and wellbeing of the individuals being studied. Since the Schladming venue is a public venue, any such observations will only be made in situations where those observed would expect to be observed and possibly recorded by strangers without their given consent. To make sure all persons passing through an area which is covered by a camera recording, signs and notifications will be placed inand around the area, notifying any individuals passing through the area of the nature and purpose of the experiment. Also, when visitors purchase an entry ticket, they will receive a flyer with information. Data Collection Any data collected during the experiment, be it visual recordings or anonymized log files from the tiled streaming application, will only be used for the purposes of the project described in this document. Subjects participating in the experiment will have access to recorded video data via the mobile application while the experiment is running. After the experiment is over, user access to this data will no longer be possible. Subjects will not have access to any log files or other data generated by the tiled streaming application on behalf of other users. Deception The participants in the experiment will not be deceived or mislead about the purpose and the general nature of the experiments. Neither will any kind of information related to the experiment be withheld from them.

Copyright TNO and other members of the EXPERIMEDIA consortium 2013-2014

18

EXPERIMEDIA

Dissemination level: PU

Withdrawal from the investigation Any participants in the experiment who have given consent and using the Tiled Streaming app will be informed about their rights to withdraw from the experiment at any given time. Since the mobile application is fully anonymous, not tied to a particular device or user, and only collects information during active use, simply stopping usage of the application will make sure that no further data is collected for that particular user. Data subjects rights will be accommodated according to the applicable data protection legislation.

4.2.

Feedback from EAB and DPB meetings

At the Ethical Advisory and Data Protection Board meeting of December 3, 2013, we discussed a number of topics. Below, a summary of the main findings and impact on the experiment is given. For the 2nd half of the second experiment, the process of notification and consent requirements consist of at least 2 steps: 1) Provide notification that recordings in the Funpark will be taking place on a particular date. The images will be captured in a setting where users are normally expected to be captured (e.g. for security reasons). This information will be announced and addressed in location itself. Also, information will be provided when purchasing tickets (additional sheet with relevant info on the areas covered, purpose, responsible entity, contact point, how long the images are kept, etc.); 2) For download and active use of application explicit consent is required. Participants have to give consent when downloading the app otherwise it will not work. Consent form will provide relevant info on the areas covered, purpose, responsible entity, contact point, how long the images are kept, etc. The following concerns were raised and recommendations were given: Participation of minors: parents can give consent or can decide not to let minors use it.

Scale of recordings: everyone entering the area is recorded. Users of the slopes are informed about it. We will consider selecting an area that will be removed from the main tracks which would eliminate random users. Moreover, those who do not wish to be recorded should be able to enjoy the facilities without too great inconvenience as they could just avoid the area. Recording individuals who did not provide their explicit consent is acceptable, when they are in the background, and not in the focus of the recordings. This allows for enabling social and sharing functions, as foreseen in the 2nd half of the second experiment period.

Downloading of the app: in the first and 1st half of the second experiment run, we will oversee and support the installation on participant's devices, e.g. those of the trainers. In the 2nd half of the second experimental run, we will allow free download of the application. When starting the application for the first time, participants will have to provide explicit consent. In the consent form they should be requested to confirm that they are at least 18 or that they have a permission
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-2014 19

EXPERIMEDIA

Dissemination level: PU

of their parents and/or guardians. This consent form, however, should not ask for the birth date. No other interference should be noticeable, i.e. the process should works similar to usual application download. Participating volunteers: we will investigate attracting participants for the 2nd half of the second experimental run beforehand, so we will have a direct interface and we can fully inform them about the experiment. Further publishing of the recordings: this possibility is not restricted to the individuals own image. There is no such technical restriction, no barriers to publish. For this reason participants who download the app should be informed in the application Terms & Conditions that they should not be uploading content which does not concern them as the main actor. Users responsibility and possible liability for such publications should be indicated. Sharing or publishing content concerning other individuals requires authorisation of the individuals in questions. Participants should be informed that they should only share/publish images of themselves. Incidental presence of others (e.g. in the background) would not consider great interference but the main subject should be the consenting individual. Data storage: data will be kept until end of May, for a short period after the end of second experimental run. This excludes content that was specifically shot for dissemination purposes, with friendly users or participants that have given their consent for such usage a priori. The content which participants choose to publish on online video platforms such as YouTube or social networks such as Facebook, is out of control of the experimenters. Videos might be kept for longer to show at demonstrations, use for valorisation and dissemination. Such dissemination will not have commercial purposes, it will be limited to closed events (industry, scientific). Participants need to be informed about this as well. Possibility to block certain content: in case of drastic or embarrassing content (skiing accident) or when someone complains that their image is processed and objects such use. Other purposes than scientific research: if such need occurs, it should be mentioned in the consent form. This would include also dissemination, valorisation purposes; as well as possible further processing by other controllers from within the consortium. If such further research is required, use of personal data from the experiments should be avoided. If this is not possible, a controller controller agreement is signed to ensure compliance with the applicable data protection legislations. Exclusivity: this concerns all the experiments at Schladming, the reviewers pointed out that they are aimed at users of an expensive ski resort who, moreover, need to have iPhones.

This leads to the following set of formal requirements Data controller: TNO, possibly together with Schladming and in a later stage also IT Innovation. Each controller will notify its respective DPA. Notification about the recordings that will be provided with the ticket, as well as the consent form for the application, should include all the relevant information (see above);
20

Copyright TNO and other members of the EXPERIMEDIA consortium 2013-2014

EXPERIMEDIA

Dissemination level: PU

Terms and Conditions of the service with the limitations and purpose to inform the users. The specification of the purpose is most likely broader than just scientific research, should cover as well dissemination and valorisation. Areas where experiment will be conducted should be selected in a way that individuals who do not want to be recorded could still use the facilities. Also, areas should be clearly indicated so no mistakes or incidental entries are made. Clear policy on participation of minors: not allowed unless consent is given by their parent or guardian. Point of contact on the site should be communicated to the participants in case of questions/ concerns/ objections. The points of contact at the later stage should be indicated in the information provided and should include a TNO and Schladming2030 contact person. Language of the communication will be English and written German.

Copyright TNO and other members of the EXPERIMEDIA consortium 2013-2014

21

EXPERIMEDIA

Dissemination level: PU

5. Experiment Design
5.1. Requirements
Our experiment mainly targets the ECC for integration and usage during the experiments. The ECC should enable the monitoring of important experiment parameters, in particular the retrieval of logging of all interaction data. That is, we need to capture the 2D coordinates of an area that a participant is looking at, so we know which parts of the video are most popular. This navigation log is created a rate of 25 samples per second, one sample per displayed image. The reason for capturing this data has been explained in section 3.6.1. Furthermore, the ECC should enable the logging of quality of service parameters, such as bandwidth and latency. Lastly, service level features should be captured, such as the user activity (how active someone is in using the system, e.g. did they just look at one point? how long did someone use the system? what features did someone use?). Note that this data may involve provenance data to recreate the actual usage by a participant. Given that the application can temporarily log the above data, the ECC should at minimum be able to pull aggregated logging data at a rate equal or lower than 25 times per second. The application may exit unexpectedly so it is necessary to send data as the application is used, not just at the end. Also, if the ECC monitoring can be done real-time, this allows for creating a live monitoring of an experiment, for instance using a live reconstruction of what people are looking at. That is, if a participant's navigation data is exchanged in real-time with the ECC, this data can be used by an adapted application to render the exact same region of the video that the participant is looking at. In a monitoring application, we can then switch between navigation data sets from the different participants. This allows us to have visually consistent observations. Ideally, this exchange of data would use web sockets. The following metrics should be implemented (with support of the ECC as far as possible): QoS o o o o Bandwidth of received streams; Interruptions in stream reception; Switching delay between resolution layers; Networking conditions, i.e. available bandwidth and minimum latency.

QoE o o o o Application Feature Call (To see which features are used where and when); Application Feature Usage Duration (How long is a certain feature used?); Navigation Feature Data; In- and Post-Experience Measurements of enjoyment and satisfaction (interesting, fun, pleased, delighted); endurability (worthwhile, watch/use again, forward, share); involvement (paid attention, captivated); usefulness (learned, increased performance, saw mistakes).

Copyright TNO and other members of the EXPERIMEDIA consortium 2013-2014

22

EXPERIMEDIA

Dissemination level: PU

Some of the parameters require that the ECC (and the ECC proxy) accepts data structures, rather than single data values. The table below shows the formatting of the QoS and QoE parameters according to the ECC Datamodel. Entity Attribute Metric Unit Metric Type Metric Group Mbps Mbps Milliseconds Milliseconds Milliseconds Action String Ratio Ratio Ratio Ratio Ratio Nominal Ratio Nominal Client QoS Client QoS Client QoS Client QoS Client QoS Client QoE Client QoE Client QoE

Client Receiver Primary stream Client Receiver Fall-back stream Client Receiver Buffertime Client Receiver Switchtime Client Receiver Ping User User User Feature XY call CurrentVideoArea

Feature XY duration Milliseconds

Notes: Primary stream indicates the bitrate of the Primary HLS stream; Fall-back stream indicates the bitrate of the Fall-back HLS stream; Buffertime indicates the time needed to buffer the Fall-back HLS stream; Switchtime indicates the time needed to switch between the Primary and the Fall-back HLS streams; Ping indicates the round-trip time for communication between the client device and the segment server; Feature XY call indicates that a user interacted with the application through feature XY; Feature XY duration indicates the duration of the usage of feature XY; CurrentVideoArea indicates the current video area at a certain point in time by timestamp, x-coordinate, y-coordinate, width and height.

Additional measurements will come from pre-post questionnaires. For example, we will inquire about the perceived visual distortion when zooming in to content captured through a fish-eye lens. We envision using the Babylon QoE measurement tool during the experiments. Use of the AVCC and SCC components is foreseen form the 2nd half of the second experimental run. Regarding the SCC component; we aim to post rendered videos on online video sharing platforms and to publish related messages on social networks. We also aim to share user navigation paths through video with friends, family or training coach. Hence, we require features to post messages to e.g. Facebook and Twitter, and to publish videos on
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-2014 23

EXPERIMEDIA

Dissemination level: PU

YouTube. Regarding the AVCC component; we aim to use the AVCC as a streaming server for delivering tiled streams. For this, we require Apple HLS input. We also aim to publish a (live) stream of user navigation to public screens, e.g. the Schladming Media Centre, and to distribute recorded streams created by users to various systems and devices. If HLS input is not available for these latter features, we will investigate transcoding to other formats, such as RTP/RTMP.

Figure 9: System architecture of the iCaCoT capture system and end-user application.

5.2.

System Architecture

The architecture for the capture system and end-user application is shown in Figure 9. The blocks depicted in red relate to physical infrastructure for capturing, such as cameras and HDSDI interfacing. Note that while the use of a 4k camera is not envisioned during the experiment, we aim to have the full system 4k-ready. Within the ingest node, the raw captured content is encoded using a mezzanine format, Motion JPEG. From the ingest node, the content is transported to a processing node, that performs the tiling, H.264 encoding and MPEG2-TS multiplexing of content. In a separate machine, temporal segmentation is performed, after which the stream segments are stored on a local web server. A client application on an iOS device is able to retrieve the stream segments for a given tile, as well as a low-resolution fall-back stream. All logging of QoS and QoE data is handled by the local ECC client, and pushed to an ECC proxy. An ECC server can then pull this data from the ECC proxy.

Copyright TNO and other members of the EXPERIMEDIA consortium 2013-2014

24

EXPERIMEDIA

Dissemination level: PU

5.3.

Content Lifecycle

A trainer can start the application and start the capture phase. This capture phase does not involve the physical capture, but identifies a period in which the trainer can later interact with the content. During this capture phase, the trainer can also indicate periods of interest using a bookmark function. In a later training phase, the trainer can then rapidly navigate to such bookmarked periods, e.g. to discuss a specific moment with a particular student. During the discussion phase, the trainer has access to the captured footage in that period and can interact with the content, e.g. to zoom in to specific parts. The trainer can also use trick-play, e.g. to pause and to highlight certain video frames, through a drawing interface. The trainer can choose to store such highlighted frames for later use. A funpark visitor can start the application and select to start the capture phase, typically before going down a slope. After his/her descent, the visitor can then interact with content to create a personalized navigational video. The visitor can choose to share this video using the social and audio-visual features. Through an admin and monitoring system, which is used as a central interface to the backend system, the processing of data and managing of content can be done easily. This includes the configuration of live monitoring, where the experimenter can select between different navigational views from users taking part in the experiment.

Copyright TNO and other members of the EXPERIMEDIA consortium 2013-2014

25

EXPERIMEDIA

Dissemination level: PU

6. Plan for Implementation

In order to achieve the experiment objectives, the work-plan evolves along three tasks, T4.11.1 T.4.11.3, as depicted in Figure 10, and two experimentation periods. Dividing the experiment into two experimentation periods allows for a further evolution of the technology components in the intermediate period. We can improve the tested components after the first round of experiments and test them again during a second experimentation period. Such improvements may be technical and in the area of user-device interaction, based on user feedback.

Figure 10: Task structure and workflow.

The planning and timelines of the various tasks and deliverables can be found in Figure 11.
Dec-2013 Nov-2013 Feb-2014 May-2014 Oct-2013 Jan-2014 Mar-2014 Apr-2014

Activity T4.x.1: Requirements and Development T4.x.2: Planning and Experimentation Experiments T4.x.3: Evaluation and Dissemination

Finish date Month 4 Month 6 Month 8

D4.x.1 D4.x.2 E1 E2 D4.x.3

Figure 11: GANTT-chart showing task and deliverable planning for the iCaCoT experiment.

Copyright TNO and other members of the EXPERIMEDIA consortium 2013-2014

26

EXPERIMEDIA

Dissemination level: PU

7. Conclusion
With the iCaCoT experiment, we focus on interactive video navigation for camera-based coaching and training for visitors and sports enthusiasts in Schladming. The goal of this experiment is to showcase the unique properties of tiled adaptive streaming, a technology that was developed in the EU FP7 project FascinatE and that allows users to navigate freely through high resolution video panoramas, while the application reduces bandwidth requirements by only sending that part of the video a user is interested in. This deliverable provides a detailed overview about the iCaCoT experiment problem statement and requirements for the Schladming Venue. This document provides EXPERIMEDIA facility operators and software components developers with requirements related to physical and digital assets and infrastructure for the implementation and execution of the iCaCoT experiment. It also provides planning information and a description of the measurement methodology. Important conclusions captured in this deliverable relate to the location and planning of the two experimental periods; the usage of and integration with the EXPERIMEDIA Baseline software components; and the requirements from an ethical and privacy perspective. The requirements and initial specifications captured in this deliverable provide a solid base for executing the experiments in 2014. We expect the experiment to have impact by raising the attractiveness of Schladming as a venue for ICT-driven sport tourism, by providing valuable input to MPEG DASH standardization, by developing technology for large-scale tests by the BBC beyond Schladming and EXPERIMEDIA, and by enhancing the potential for exploitation of tiled streaming technology.

Copyright TNO and other members of the EXPERIMEDIA consortium 2013-2014

27