D4.6.3 CONFetti Experiment Results and Evaluation v1.0 PDF

D4.6.
3 CONFetti Experiment Results and Evaluation

2013-10-29
Sergiusz Zieliski (Pozna Supercomputing and Networking Center)
This document presents the results of the CONFetti experiment which aimed to investigate the possible applications of FMI (Future Media Internet) technologies in the improvement of the sports training process. The experiment was performed in the CAR (Centre d'Alt Rendiment) venue and employed technologies such as videoconferencing, stereoscopy, augmented reality and motion tracking.
www.experimedia.eu
EXPERIMEDIA
Project acronym EXPERIMEDIA
Dissemination level: PU
Full title Experiments in live social and networked media experiences Grant agreement number 287966 Funding scheme Large-scale Integrating Project (IP) Work programme topic Objective ICT-2011.1.6 Future Internet Research and Experimentation (FIRE) Project start date 2011-10-01 Project duration 36 months Activity 4 Experimentation Workpackage 4.6 Remote and collaborative training for acrobatic sports Deliverable lead organisation Pozna Supercomputing and Networking Center Authors Sergiusz Zieliski (Pozna Supercomputing and Networking Center) Reviewers Diego Esteban (ATOS) Version 1.0 Status Final Dissemination level PU: Public Due date PM24 (2013-09-30) Delivery date 2013-10-29
Copyright Pozna Supercomputing and Networking Center and other members of the EXPERIMEDIA consortium 2013
EXPERIMEDIA
Table of Contents
1. 2. 3. Executive summary ............................................................................................................................ 4 Introduction ........................................................................................................................................ 5 Experiment architecture .................................................................................................................... 7 3.1. Architecture overview .............................................................................................................. 7 3.1.1. HD videoconferencing......................................................................................................... 7 3.1.2. 3D............................................................................................................................................ 7 3.1.3. Motion tracking ..................................................................................................................... 7 3.1.4. Visualisation ........................................................................................................................... 8 3.1.5. Integration with the Kinect cycling experiment ............................................................... 8 3.1.6. Experiment Content Component ...................................................................................... 9 3.1.7. Audio Visual Content Component .................................................................................... 9 3.2. 3.3. 4. 4.1. 4.2. 5. 5.1. 5.2. 5.3. 6. Equipment.................................................................................................................................. 9 Ethics and privacy ................................................................................................................... 10 First experiment run ............................................................................................................... 11 Second experiment run .......................................................................................................... 12 Questionnaires ......................................................................................................................... 14 Impact of QoS on QoE ......................................................................................................... 14 Dissemination .......................................................................................................................... 15 Questionnaire ............................................................................................................... 17
Experiment execution ...................................................................................................................... 11
Results ................................................................................................................................................ 14
Conclusions ....................................................................................................................................... 16
Appendix A.
EXPERIMEDIA
1. Executive summary
This document presents the results of the CONFetti experiment which aimed to investigate the possible applications of FMI (Future Media Internet) technologies in the improvement of the sports training process. As the relationship between the coach and the athletes is irreplaceable, the goal is not to substitute it, but to augment the availability, convenience and effectiveness of the process basing on this relationship. The technologies that were utilised include HD videoconferencing, stereoscopy, motion tracking and augmented reality. The experiment was held at the CAR (Centre d'Alt Rendiment) venue, allowing the experimenters to evaluate their system in a realistic target environment. Section 2, which is an introduction, explains what the experiment's premise was and what the different stakeholders wanted to obtain through performing it. Section 3 describes the experiment architecture in terms of software and hardware as well as the legal issues associated with it. Section 4 provides information about the two runs of CONFetti, while Section 5 summarises its results. Section 6 gives a conclusion of the document and experiment.
EXPERIMEDIA
2. Introduction
The CONFetti experiment aimed to investigate the possible applications of FMI technologies in the improvement of the sports training process employed at the CAR venue. The technologies that were utilised included HD videoconferencing, stereoscopy, motion tracking and augmented reality. The experiment employed a system allowing the coach to hold a training session without the need for his presence in the training hall. He connected remotely with his protgs located in the hall using a high definition (HD) videoconferencing tool. The system enabled him to observe the athletes performing their gymnastics routines and, afterwards, to give them advice while reviewing an instant replay with slow motion capabilities (Fig. 1). Additionally, the coach has been able to review and present archival footage of athletes recorded with stereoscopic (3D) HD cameras. This footage also contained a 3D model of a human body reflecting the athlete's movements collected via motion tracking. The stream with the superimposed model was treated as one of the sides of the videoconference for both the athletes and the coach to view.
Figure 1: A coach and gymnast reviewing a pommel horse performance stereoscopic replay
The experiment's goals from the venue's point of view were to evaluate the possible benefits offered by the various FMI technologies used. The training process could get improved in the following ways: The availability of coaches for sessions is incremented as they can be held even when the coach is not present at the venue. The convenience for the coaches is raised thanks to the possibility of holding sessions from remote locations. The coach has access to additional motion tracking and sensor data concerning the athletes' performance, on which he can base his decisions. The transfer of knowledge and instructions is more efficient with the use of archival footage and 3D model teaching aids.
5
EXPERIMEDIA
The use of new technologies like augmented reality or stereoscopy may appeal to young athletes and make the sessions more interesting and, in turn, more beneficial.
The experimenters carried out the experiment in order to investigate the following issues: The feasibility of installing and utilising a setup for stereoscopic HD videoconferencing and augmented reality in a real-life training facility User acceptance level (concerning both coaches and athletes) of the FMI functionalities provided by the system deployed in a sport training setting Performance of a platform combining HD videoconferencing with stereoscopic video and remotely rendered 3D models generated basing on motion tracking
EXPERIMEDIA
3. Experiment architecture
The CONFetti experiment design involved constructing the software platform, complementing it with a number of hardware elements and solving some organisational, scheduling, logistics and legal issues.
3.1.
Architecture overview
The CONFetti platform combines a number of FMI technologies and was integrated with a few external components, all of which are mentioned below.
3.1.1.
HD videoconferencing
This component's main task is to enable a stereoscopic HD videoconferencing connection between the athletes in the training room and the remote coach. Additionally, it allows the coach to review and present the archival footage of previous athletes' performances. There are two instances of the videoconferencing client, one running on a machine at the coach's site, the other in the training hall at CAR. The videoconferencing solution adopted for the CONFetti experiment is based on HDVIPER, an open and scalable HD videoconferencing platform developed in the course of the CELTIC initiative with a substantial participation of PSNC. The aim of the HDVIPER project was to provide a converged platform for High Definition videoconferencing (point-to-point and multi-point transmissions), to be integrated in a PC environment (pre-commercial prototype) and to be tested in residential VC, healthcare, business VC and distance education scenarios. Thanks to its extendable architecture the platform's functionality can easily be broadened with additional services apart from HD video and audio connectivity. The architecture is based on SIP (Session Initiation Protocol), which is very widespread in Internet multimedia applications. Its popularity makes possible the platform's compatibility with a wide array of solutions including software phones, smart phones and videoconferencing terminals. HDVIPERs architecture is compatible with various client applications, but for the purposes of the experiment one solution had to be selected and appropriately extended. The chosen software is Minisip, an open source SIP User Agent developed by the Royal Institute of Technology (KTH) in Stockholm and a community of developers including PSNC employees.
3.1.2.
3D
The 3D functionality utilised in CONFetti is based on stereoscopy. A number of dual-lens cameras are used to acquire footage of the athlete performing the routine, the coach and the athletes gathered by the training hall VC terminal. The stereoscopic signal is transmitted through the videoconference in the Side-by-side format and displayed on 3D enabled TVs. The experiment participants are required to wear dedicated glasses in order to view the 3D video feed. Additionally, a 3D transcoder component was developed with the ability to convert the 3D signal to other formats than SbS.
3.1.3.
Motion tracking
The motion tracking functionality in the CONFetti experiment was realised in cooperation with STT, the EXPERIMEDIA partner responsible for the 3D Acrobatic Sports experiment. The
Copyright Pozna Supercomputing and Networking Center and other members of the EXPERIMEDIA consortium 2013 7
EXPERIMEDIA
first run of the experiment was used as an opportunity to record gymnastic performances and, in parallel, gather motion tracking data of the same performance, allowing to create 3D models of the athletes bodies animated using this data. During CONFettis first run inertial Bluetooth sensors were utilised, whereas for the second run they were replaced by reflective markers (Fig. 2).
Figure 2: The capturing of a pommel horse routine with motion capture
3.1.4.
Visualisation
The visualisation component is used to render the 3D models that are superimposed on the archival video material. The input data for the animation rendering process are models and sensor events acquired by motion tracking done by the STT experiment. The software used for this component is the Vitrall visualisation system designed and implemented at PSNC. Vitrall is a distributed, web based visualisation system that is capable of remote rendering of complex 3D content coming from various sources, e.g. scientific data or multiple 3D model formats. It is designed to efficiently utilise multi-GPU and CPU-GPU hybrid server installations. Thanks to highly efficient remote server rendering even a less capable client is able to efficiently display and manipulate large 3D datasets in real-time. The system can also be used in collaborative environments, where many users can simultaneously interact, modifying the 3D content in real-time. Vitralls high-performance remote visualisation capabilities have been successfully demonstrated at various international conferences.
3.1.5.
Integration with the Kinect cycling experiment
The CONFetti platform was integrated with the system utilised by CERTH in the Kinect cycling experiment. This allowed to hold a 3D HD videoconference between the cycling training spot and a remote location with statistical data from CERTH's system superimposed onto the stereoscopic videoconference stream (Fig. 3).
EXPERIMEDIA
Figure 3: Stereoscopic videoconference held during the CERTH cycling experiment
3.1.6.
Experiment Content Component
The ECC's role is to gather monitoring QoS data from the other components that register as sources of such data. Since all the software components of the CONFetti setup are written in C++, a C++ client for the Experiment Content Component was needed. An API delivered by the component's creator enabled the experimenters to implement the required client functionality in the system and use it to gather usage statistics during the second run of the experiment.
3.1.7.
Audio Visual Content Component
The role of AVCC in the CONFetti experiment was to serve as a video repository. Videos of athletes performing the gymnastic routines with 3D models superimposed were uploaded to an AVCC instance and can be streamed to the Minisip videoconferencing client in order to review them during the training session, giving the coach a possible source of training aids.
3.2.
Equipment
Apart from the software platform, the CONFetti experiment utilised a set of hardware components that had to be transported to CAR for each experiment run. The main hardware elements were: 3 stereoscopic cameras for capturing footage of the athlete performing the training routine, the coach and the athletes gathered by the VC terminal 2 PCs with Blackmagic Decklink capture cards for receiving the HD stereoscopic signal A NewTek 3Play replay machine for instant replay and frame by frame analysis functionality 2 3D TVs for displaying the videoconferencing signal in the training hall and in the coach's room A video recorder capable of outputting stereoscopic materials from flash cards
EXPERIMEDIA
3.3.

Ethics and privacy

Obtaining consent for participation in the experiment from coaches and athletes Storing and processing video material including the participants Storing and processing animations of the athletes' bodies Storing and processing questionnaires filled by the participants
The key ethics and privacy issues to consider for the CONFetti experiment were:
The consent from the participants was obtained by CAR representatives. As for the body animations and questionnaires, they are anonymous and as such do not pose a problem in regard to privacy. The key issue proved to be the video materials used both for the creation of training aids and for the production of the promotional videoclip. Discussions between representatives of the venue, the experimenters and the consortium partners responsible for Privacy Impact Assessment have been held during a number of project meetings and followed up with an e-mail exchange during which a questionnaire was filled by both the experimenters and the venue representatives in order to complement the PIA-related experiment information that was delivered in D4.6.1. After gathering the necessary information representatives of the partner responsible for PIA proposed an action plan regarding personal data flows in the CONFetti experiment. The main point of this plan was the need to sign a controller-processor contract between the venue and the experimenting institution (CAR and PSNC). Such an agreement has been drafted to resolve the mentioned issues.
10
EXPERIMEDIA
4. Experiment execution
Two runs of the CONFetti experiment were performed, in May and September 2013. The original schedule assumed the second run would be done between June and August, however June would be too soon after the experiment's first run and the availability of participants at the venue during July and August was problematic.
4.1.
First experiment run
The first run of CONFetti, done on May 15-16th 2013, included an HD 3D videoconference with instant replay and frame-by-frame functionality. Archival materials recorded in January 2013 were also available for the coach to use, although they did not yet contain 3D models of the athletes' bodies. The opportunity was used to record gymnastic performances in parallel with tracking the athletes' motions performed by STT. This allowed to subsequently render the 3D models for use in the future. Additionally, all the experiment's participants were asked to fill questionnaires concerning their views on the system, its different functionalities and the difference it made in the training session.
Figure 4: The coach room during a CONFetti experiment run
The equipment involved was installed in two rooms in CAR: the training hall and an office located in a different building, which played the role of the remote coach's post (Fig. 4). This post was equipped with a 3D camera on a tripod, a 3D-capable TV, a recorder capable of outputting stereoscopic materials from flash cards and a PC with two Blackmagic DeckLink capture cards. The Minisip videoconferencing client was installed on this computer, with the camera and the aforementioned recorder connected as sources of video streams. The recorder played back performances of the gymnasts recorded in January. The videoconferencing client installed on the coach's terminal maintained a connection with the training hall, where the setup was a little more complicated. It consisted of two 3D cameras on tripods, two 3D TVs, a NewTek 3Play replay machine and a PC with two DeckLink cards and Minisip installed. The signal sources for Minisip were the cameras - one of them was directed at the athletes and coaches taking part in the videoconference from the training hall, whereas the other was directed
EXPERIMEDIA
at the gymnast performing his routine on the pommel horse. The signal from the latter camera was transformed on the way by the replay machine, which made it possible to have instant replays with fluent speed control and frame-by-frame analysis as well as straight forwarding of the live feed to the videoconference. The VC signal was simultaneously displayed on two 3D TVs, one for the on-site coach and one for the athletes. Altogether, four stereoscopic HD streams were simultaneously sent through the videoconference: the coach and the archival materials from one side; the athletes discussing with the coach and the athlete performing the pommel horse routine (or the replay) from the other side. The choice of displayed signal was independent and free on both sides. During a single training session an athlete performed the routine on the pommel horse, which was observed by two coaches: one present in the training hall, the other one remote. Subsequently, the gymnast joined his colleagues gathered by the VC terminal, where the replay of his performance was displayed. The coaches consulted each other, commented on the routine observed in the replay and gave advice to the athlete. Additionally, the remote coach had the possibility of presenting archive video materials with performances by the same gymnasts from 4 months before.
4.2.
Second experiment run
The second run of the CONFetti experiment was performed on 16-18th September 2013. This time, apart from exposing the participants to the functionalities present in May, the 3D models rendered basing on motion tracking data collected by STT were used. The animated models rendered by the Vitrall system were superimposed onto footage of athletes performing that was recorded during the first run. The animation of the model corresponded with the athlete's movements to give the viewers a more comprehensive overview of the performance (Fig. 5).
Figure 5: Footage of a pommel horse session with the skeleton model superimposed
Additionally, the CONFetti platform was integrated with the system utilised by CERTH in the Kinect cycling experiment. The biomechanical data generated by the system was superimposed
EXPERIMEDIA
on the stereoscopic videoconferencing signal allowing the remote coach to observe it alongside the feed of the cyclist. Thanks to the videoconferencing connection they were able to communicate and exchange remarks regarding the training session. The opportunity was also taken to record high quality 4K video materials of the venue and experiments in order to produce a videoclip for the purpose of promoting the experiment and project. A short interview was also recorded with the CTO of CAR, Josep Escoda.
13
EXPERIMEDIA
5. Results
5.1. Questionnaires
The experiment was received very enthusiastically by both the coaches and the athletes, as well as other CAR employees who visited both posts very often. The participants of the experiment were asked to answer questionnaires prepared in cooperation with CAR psychologists, which helped in obtaining information on usability of the CONFetti system in this scenario as well as ideas for possible improvements. Analysis of the answers given shows that the technology might prove useful in this scenario. Both coaches and athletes stated that the instant replay and 3D features allowed them to evaluate the performances in a more detailed way. The videoconferencing ability was appreciated by the coaches who would gain the benefit of more flexible scheduling from it. It became apparent that a person overlooking the athletes in the training room is still necessary, as one of them expressed concern what would happen if he got injured with no coach around. Another concern was that the 3D glasses needed to view the 3D signal were a slight annoyance, as they had to be put on and taken off constantly. This might be solved with the perfection and popularisation of glassless 3D technologies. The experiment has also shown other limitations of the technology in this scenario. The sports discipline needs to be fairly static in terms of space in order to fit in the frames of all the cameras. Also, 3D technology requires a steady distance between the camera and the main plane of the scene, at least in its more basic form. Those obstacles can be overcome with more staff and more expensive equipment, but if the essential setup is to be used it requires some preparation beforehand and puts some limitations on the usage areas. The most interesting results drawn from the questionnaires were: The vast majority of participants (12 out of 14) strongly agreed that the system was easy to use All of them found the video quality to be good The fluency was described as smooth enough to observe the athletic performance, although the answers were not as strong as in the case of the question about quality 3 people noticed the delay in the videoconferencing signal 5 people thought it was disturbing to use 3D glasses The audio quality was described as good The instant replay and frame by frame functionality was found to be useful 1 person thought that the equipment setup disturbed the training session The 3D video allowed the participants to review the performance more thoroughly and better visualise the remarks of the coach
5.2.
Impact of QoS on QoE
To facilitate a deeper evaluation of the stereoscopic technology during CONFetti runs, a small part of the sessions was held in 2D for comparison. All the signals in the experiment were sent in the 1080p25 format. To get the coaches' opinion on this matter, materials in the 720p50, 1080p50, 1080p25 (Full HD) and 576p50 (SD) formats were also presented to them. Results of
EXPERIMEDIA
this study show that the difference between the Full HD and 720p resolutions is not noticeable in these circumstances, whereas the difference between 720p and SD was noticed, as well as the one between 50 and 25 frames per second. The conclusion would be that in the case of the setup that was used in CONFetti (specifically the screen sizes and viewing distances), the 720p resolution provides a high enough quality. The fluency of the movement provided by the 50 fps framerate is more important in this scenario than the additional details provided by Full HD. This is especially significant because some equipment components do not allow to combine both, effectively forcing the users to choose between resolution and framerate.
5.3.
Dissemination
Disseminating the results of research projects is a crucial factor in being part of the scientific community. That is why the experimenters have used a number of opportunities to spread information about CONFetti:
Figure 6: Visitors of Researchers' Night 2013 at the CONFetti stand
A paper titled "Stereoscopic videoconferencing with augmented reality in technology enhanced sports training was submitted to the eChallenges 2013 conference's Technology Enhanced Learning thematic area and a presentation was given at the conference itself EXPERIMEDIA and CONFetti were mentioned in presentations during the i3'2013 conference as well as the FI-PPP InfoDay and Brokerage Event in Pozna The CONFetti platform was used in a stand at Researchers Night 2013 in Pozna (Fig. 6) A promotional videoclip of CONFetti and EXPERIMEDIA was produced
15
EXPERIMEDIA
6. Conclusions
The analysis of the questionnaires and the QoS data measured during the experiment runs shows that the system is suitable for application in a training venue, both performance and functionality wise. The feedback that was gathered also contains useful remarks concerning possible enhancements and fine-tuning of the systems parameters. The experiments results show that the deployment of a system offering CONFettis functionalities in CAR would be beneficial for all the stakeholders involved. The athletes found it fun and interesting to use and also stated that the instant replays in slow motion and 3D help them review their performance and understand the coachs remarks. For coaches, the additional functionalities, the high quality of the VC signal and the stereoscopy make the remote training session a valid alternative to personal interaction, allowing them to save time and be more flexible in scheduling. For the venue, the system could allow to decrease the costs of the training program and increase the availability of coaches, which is especially important in the times of austerity. The system created during the CONFetti experiment proved to be useful for sports training applications. The feedback gathered from both the athletes and the coaches verifies that, while additionally giving some interesting remarks about possible enhancements. The venue also found it beneficial to make use of the systems possibilities in the training process. The tests have also proven that it is feasible to set up such a system in a real-life training hall and that its performance is satisfying for the purposes evaluated in the scenario.
16
EXPERIMEDIA
Appendix A. Questionnaire
The below questionnaire was used to gather participant feedback during after the CONFetti experiment runs. It was presented to the coaches and athletes in Spanish. The following statements could be rated using a scale of 1-5, where: 1 I strongly disagree 2 I disagree 3 I don't know 4 I agree 5 I strongly agree A 1. It was easy to understand how the system works.
2.
The video quality was good.
17
EXPERIMEDIA
3. The high definition gives me more details that are important for evaluation of the performance.
4. The fluency of the video was smooth enough for me to observe the athletic performance.
5.
The video signal from the other side was delayed.
18
EXPERIMEDIA 6. It was a disturbance to use the 3D glasses.
7. The audio quality was good and allowed me to hear the other side of the videoconference clearly.
8.
It was useful to use instant replay and frame by frame functionality.
19
EXPERIMEDIA
9. I think it is useful to have the possibility of holding training sessions through videoconferencing.
10.
The technology setup disturbed my concentration during the training session.
B 1. I think seeing the video is useful for learning or perfecting the technique.
20
EXPERIMEDIA
2. 3D observation helps me to see in more detail the position of my body during the execution compared with a standard video image.
3. 3D observation allows me to be more aware of the body extension while performing compared with a standard video image.
4. The 3D observation allows me to be more aware of the inclination of the body relative to the rack during the performance compared to a standard video image.
21
EXPERIMEDIA
5. 3D observation allows me to better be aware of the action of the arms during execution (Extension-abduction) compared with a standard video image.
6. 3D observation allows me to be better aware of rhythm and exercise execution speed compared to a standard video image.
7.
3D observation helps me relive the sensations I felt during the exercise.
22
EXPERIMEDIA 8.
Usually I use visualization to improve the execution of technical motions.
9. I think 3D observation I would improve the visualization of the execution of the exercise.
How do you think the system could be improved? 3D without glasses More cameras to add different angles of the performance A coach still has to be around in case of injury
23

D4.6.3 CONFetti Experiment Results and Evaluation v1.0 PDF

Încărcat de

Informații document

Titlu original

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

D4.6.3 CONFetti Experiment Results and Evaluation v1.0 PDF

Încărcat de

Drepturi de autor:

Formate disponibile

D4.6.

3 CONFetti Experiment Results and Evaluation

Sergiusz Zieliski (Pozna Supercomputing and Networking Center)

Experiment execution ...................................................................................................................... 11

Figure 2: The capturing of a pommel horse routine with motion capture

Integration with the Kinect cycling experiment

Figure 3: Stereoscopic videoconference held during the CERTH cycling experiment

Experiment Content Component

Audio Visual Content Component

Ethics and privacy

First experiment run

Figure 4: The coach room during a CONFetti experiment run

Second experiment run

Impact of QoS on QoE

Figure 6: Visitors of Researchers' Night 2013 at the CONFetti stand

The video quality was good.

The video signal from the other side was delayed.

EXPERIMEDIA 6. It was a disturbance to use the 3D glasses.

It was useful to use instant replay and frame by frame functionality.

The technology setup disturbed my concentration during the training session.

3D observation helps me relive the sensations I felt during the exercise.

Usually I use visualization to improve the execution of technical motions.

S-ar putea să vă placă și