D2.1.1 First EXPERIMEDIA Methodology v1.02

D2.1.
1 First EXPERIMEDIA Methodology

2012-03-01
Marie-H.Gabriel (FDF), Eric Seulliet (FDF), Peter Ljungstrand (Interactive Institute), Niels Vandezande, Eleni Kosta (KU Leuven), Michael Boniface (IT Innovation)
This document describes the experimental methodology for social and networked media experiments targeting the EXPERIMEDIA facility. The document provides experimenters with best practice approaches that build on those used by Future Internet testbeds (FIRE) and user centred facilities (Living Labs). The document will show how to design experiments considering the roles of different stakeholders (experimenters, testbed providers and developers). The methodology will consider how ethical concerns need to be addressed throughout the experiment lifecycle in line with EXPERIMEDIA ethics management procedures.
www.experimedia.eu
EXPERIMEDIA
Dissemination Level: PU
Project acronym EXPERIMEDIA Full title Experiments in live social and networked media experiences Grant agreement number 287966 Funding scheme Large-scale Integrating Project (IP) Work programme topic Objective ICT-2011.1.6 Future Internet Research and Experimentation (FIRE) Project start date 2011-10-01 Project duration 36 months Activity 2 Construction Workpackage 2.1 Blueprint Architecture Deliverable lead organisation FDF Authors Marie-H. Gabriel, Eric Seulliet (FDF) Peter Ljungstrand (Interactive Institute) Niels Vandezande, Eleni Kosta (KU Leuven) Michael Boniface (IT Innovation) Reviewers Angelos Yannopoulos (ICCS) Simon Crowle (IT Innovation) Version 1.02 Status Final Dissemination level PU: Public Due date PM3 (2011-12-31) Delivery date v1.0 2012-03-01; v1.01 2012-03-12; v1.02 2012-10-15
Version Changes v1.0 First release v1.01 Corrected authors v1.02 Updated front page and meta-data table
Copyright FDF and other members of the EXPERIMEDIA consortium 2012
EXPERIMEDIA
Table of Contents
1. 2. Executive Summary............................................................................................................................ 3 Introduction ........................................................................................................................................ 4 2.1. 2.2. 3. 3.1. 3.2. 4. 4.1. 4.2. Purpose ....................................................................................................................................... 4 Scope ........................................................................................................................................... 4 Challenges for Evaluation of FMI Systems with Users ....................................................... 9 Challenges of Dynamic Multidisciplinary and Multicultural teams ................................... 9 Introduction ............................................................................................................................. 11 Phases of the Value Impact Assessment ............................................................................. 11
Methodological Principles and Challenges ..................................................................................... 7
Value Impact Assessment Frame (VIA) ....................................................................................... 11
4.2.1. VIA Phase 1: Value Opportunity Assessment................................................................ 12 4.2.2. VIA Phase 2: Value Opportunity Validation and Macro Modelling Phase ................ 14 4.2.3. VIA Phase 3: Industrialisation Assessment Phase ......................................................... 15 4.3. 5. 5.1. 5.2. VIA Review Board Processes ................................................................................................ 15 Introduction ............................................................................................................................. 16 Privacy Impact Assessment (PIA) ........................................................................................ 17 Methodological Approach towards Legal and Ethical Requirements ...................................... 16
5.2.1. From a Privacy Impact Assessment on RFID ........................................................... 17 5.2.2. to a general Privacy and Data Protection Assessment? ............................................ 18 5.3. 5.4. PIA methodology development ............................................................................................ 19 EXPERIMEDIA PIA ............................................................................................................ 21
5.4.1. PIA Phase 1: Preparation ................................................................................................... 22 5.4.2. PIA Phase 2: Pre-assessment ............................................................................................ 23 5.4.3. PIA Phase 3: Risk Assessment .......................................................................................... 25 5.4.4. PIA Phase 4: Documentation and Reporting ................................................................. 28 5.4.5. PIA Phase 5: Review .......................................................................................................... 29 5.5. 6. PIA Outcome .......................................................................................................................... 30 Conclusion ......................................................................................................................................... 31
EXPERIMEDIA
1. Executive Summary
This document describes the first version of guidelines for conducting experiments at EXPERIMEDIA facilities. The objective is to provide experimenters with a set of methodologies that can ensure robust experiment design and improved understanding of the impact of results. The document is especially important for experimenters wanting to submit proposals to EXPERIMEDIA open calls as it provides guidelines and expectations for assessment of value impact, technical impact and privacy impact. Initial elements of the methodology are expected to form part of submitted proposals. The document outlines the primary challenges and principles for design of social and networked media experiments using Future Media Internet (FMI) technologies. The need for a balanced and iterative approach that considers the interplay between FMI technologies, venue ecosystems and users participating within experiments will be essential for EXPERIMEDIA's success. Special attention is necessary for engagement of users during the assessment of experience and privacy impact. The dynamic, multidisciplinary and multicultural nature of teams within the project raises additional challenges that need to be addressed both from a rationalized design perspective and a collaborative viewpoint. This initial guideline provides a meta-method frame for experiments based on Value Impact Assessment (VIA) and Privacy Impact Assessment (PIA). VIA focuses on defining, measuring and assessing business release value at specific milestones during the lifetime of an experiment. The business release value is defined in terms of Key Performance Indicators derived from Quality of Service (QoS) and Quality of Experience (QoE) metrics. VIA is organised into three phases each designed to incrementally move towards industrialisation and large scale trials. PIA focuses the formal assessment of privacy risks and ensures that experiments conducted using the EXPERIMEDIA facility take into account ethical concerns, mainly relating to the protection of personal data and the privacy of the users involved in such experiments. PIA has five different phases: preparation, pre-assessment, risk assessment, documentation and reporting, and review. The outcome of the PIA assessment will report on identified risks and mitigation strategies to be implemented to ensure that no residual risks remain unaddressed. The meta-method frame will now be elaborated through use by the driving experiments of EXPERIMEDIA at Schladming Ski Resort, Multi-Sport High Performance Centre of Catalonia (CAR) and Foundation of the Hellenic World (FHW). Specific methods will be defined for the desired experience patterns within each of the experimental scenarios. These patterns of experience will provide methodology templates for future experiments to be conducted at the EXPERIMEDIA facility.
EXPERIMEDIA
2. Introduction
2.1. Purpose
This document describes the first version of guidelines for conducting experiments at EXPERIMEDIA facilities. The objective is to provide experimenters with a set of methodologies that can ensure robust experiment design and improved understanding of the impact of results. The document is especially important for experimenters wanting to submit proposals to EXPERIMEDIA open calls as it provides guidelines and expectations for assessment of value impact, technical impact and privacy impact. These elements of the methodology are expected to form part of submitted proposals. Although the primary audience are experimenters, the ability of EXPERIMEDIA to deliver experimental insight is deeply connected to facility capabilities. Therefore, a secondary objective is to provide facility developers with a view on how experiments are being approached in terms of process, data and necessary insight. The expectation is that this will lead to technical requirements for the facility itself.
2.2.
Scope
This document outlines principles of the methodological approach for conducting experiments at EXPERIMEDIA facilities. The document introduces the challenges for executing user centric social and networked media experiments and an outline of the 'toolbox' of methods and approaches expected to be used. This is the first version of the EXPERIMEDIA methodology which will be further refined throughout the project in D21.4 (M15/Dec-12) and D21.7 (M27/Dec-13) based on lessons learnt and best practice. Our aim is to assess and define principles, policies and processes as to be able to choose robust experimental approaches and analytical methods depending on a typology of rich and live sociotechnical interactions. The outcome will be a value based assessment methodology based upon a critical set of sub-scenarios, with defined measurement criteria, so to optimise scenario deployment on a large scale, and minimise risks and time to develop new scenarios. A particular focus is placed on the ethical aspects and compliance; this is a key issue in experiments we shall lead with user participation and will naturally balance the value based assessment methodology we aim to develop. The current version of the document focuses on a meta-method level to provide a high level view of the experimental evaluation process. Iterations of the meta-method through the experiments during the project lifetime will enable us to improve its application and definition of limits. The methodology will initially be verified and validated by EXPERIMEDIA's driving experiments at Schladming Ski Resort, Multi-Sport High Performance Centre of Catalonia (CAR) and Foundation of the Hellenic World (FHW), and then will be used by experiments funded by open calls.
EXPERIMEDIA
Figure 1. The relationship of methodology to EXPERIMEDIA activities
Figure 1 shows the relationship between the methodology and other EXPERIMEDIA activities. The methodology forms part of a collection of documents that define a generalised blue print architecture for Future Media Internet (FMI) testbeds, as defined by Activity 2 "Construction". Others in this set include Requirements and Scenarios1 and Blueprint Architecture. The methodologies selected by an experiment will depend on the desired experience patterns of experiments as expressed within the requirements and scenarios. We expect guidelines to be defined for different pre-defined patterns of interaction and experience under study. At the core of the methodology are three main elements related to business, user and technology impact. For business and user impact an experimenter must consider how the proposed innovations impact venue ecosystems. It is within these ecosystems where impact must be explored and hopefully demonstrated. The ecosystems are defined by Activity 3 "Operations" and vary between each facility. The ecosystems include descriptions of stakeholders, assets, interests, governance policies, business models and relationships. A special consideration for EXPERIMEDIA is that experiments include human participants. As such ethical oversight is required and all experiments must undergo ethics review by the EXPERIMEDIA Ethics Advisory Board (EAB). The methodology includes a specific process to assess privacy impact on experiment participants. We expect that an initial privacy impact assessment is completed within the preliminary experiment definition phase prior to submission to the EAB. When considering technology impact, an experimenter must analyse how a component under test can be integrated with technology from the EXPERIMEDIA Toolbox. This analysis must also take into account the broader requirements of its role in the production of an FMI system for evaluation, and what disruptions to existing technological solutions may occur as a result. Technical impact must be
http://www.scribd.com/doc/79825559/D2-1-2-Requirements-and-Scenarios-v1-01
EXPERIMEDIA
considered at three levels: conceptual capability, EXPERIMEDIA toolbox and venue infrastructure.
EXPERIMEDIA
3. Methodological Principles and Challenges

EXPERIMEDIA will provide testbed services for live, complex, social-technological experimentation of networked media systems. The experiments will be based upon a set of scenarios selected to explore new and innovative types of live interaction between individuals and communities, with many parameters such as localisation, personalised content creation and information exchanges. EXPERIMEDIA needs to provide meta-methods that tell experimenters how to design and conduct an experiment within this environment in a way that can deliver real insight. The approach must be designed for simplicity and usability due to the diversity of teams. It is important that the meta-method allows experimenters to engage with the diversity of people (e.g. those from technological, commercial and research backgrounds) effectively from a set of best practice. The approach taken for each experiment must also allow experimenters to determine the specific methodological tools; what to measure; levels of confidence and certainty and when such techniques need to be applied within the experimentation course. We call the scope of the methodology the meta-method frame which aims to enable decision making considering the requirements to: capture a wide range of quantitative and qualitative data from many sources analyse qualitative data from multi-faceted sources in a variety of formats from within the chosen scenario contexts observe interactions between individuals and the computer system in order to evaluate the most valuable experiences for the user, and the potential business value the scenario offers stakeholders (e.g. the venues) provide feedback to users and stakeholders on the observations and measurements as to cross fertilize their experiences and enrich the value analysis process redefine and iterate the scenario testing, as to further assess the scenario sampling quality prior to first scaling these scenarios to larger numbers of people, then up to live mass audience events
The success of EXPERIMEDIA is contingent on the ability of experiments to deliver impact within venue ecosystems primarily in terms of increasing quality of experience (QoE) for endusers and maximising the business value for themselves or other service providers. An experimenter's ability to achieve this goal resides in the core capability offered by EXPERIMEDIA. As such the meta-method should emphasize observing and analysing those users' motivations, individually and collectively, from the concept design phase to the prototype testing, and provide user feedback at each step of the new service development. In parallel, a value assessment for the venues needs to be developed with the ultimate goal of building a step by step experimental process which effectively identifies and evaluates business value based upon QoE for end users, associated costs and resources. Within this process, a key objective is to envision future extrapolations and create a value assessment framework that would ultimately define the assessment of FMI business models in such a way as to mitigate risks prior to their scaling up. We aim for our approach to fit most experiments, based upon a defined but extensible typology of experiments that derive from behavioural and interaction patterns. The
EXPERIMEDIA
patterns are expected to be characterised by sampling from multi-faceted data sources (qualitative and quantitative) which will be collected through the experimentation process. The typology of experiments shall be based on a taxonomy of experiments' items. In other words, a classification of experiments' basic measures based upon pattern characteristics (QoE, QoS parameters, risks and benefits for stakeholders within venue ecosystems). The meta-method shall be framed to balance user value, business value, and technology value. This balanced and iterative approach is key to EXPERIMEDIA success; it addresses the need for experiments to consider the interplay between FMI technologies, venue ecosystems and users participating within experiments. This is illustrated hereunder in Figure 2.
Figure 2. The nature of a break-through balanced innovation process, as an iterative and collaborative interaction between business, user and technology value2
User Value: indicators that contribute to user satisfaction and maximising experience as defined by a level of QoE Business Value: indicators that contribute to business success as defined by key performances indicators (KPIs) Technology Value: indicators related to system performance necessary to create the desired level of QoE as defined by a level of QoS
These values are all linked and the meta-method needs to provide mechanisms to robustly define, model and correlate these different types of metrics.
Source: Interviews with Herman DHooge (Innovation Strategist, Desktop Platforms Group) and Tony Salvador (Design Ethnographer, Emerging Markets Business Unit), Intel, (as mentioned in the brochure User-Driven Innovation Context and Cases in the Nordic Region published in June 2008 by the Nordic Innovation Center).
2
EXPERIMEDIA
The temporal horizon for the meta-method is expected to be from today to the horizon of three years. Of course, for each experiment this will be dependent on the nature of the innovation, maturity of technology and related barriers to adoption. Due to the fact that EXPERIMEDIA systems will be deployed at venue ecosystems with real users, it assumes that such systems will have a certain level of technology maturity that allows for effective validation. Therefore we have made an initial judgement that a three year horizon would be a good starting point.
3.1.
Challenges for Evaluation of FMI Systems with Users
EXPERIMEDIA requires the development of a solid methodological base (i.e. the fundamental principles, policies and processes) shared by all stakeholders as to enable initial experimentation in the first place, and subsequently to maximise the scaling up potential of those experiments for their future industrialisation. The main challenge in respect to FMI Systems with users is:
Conciliating an observation, ground based qualitative approach (Quality of Experience) with objective and quantitative data measures (Quality of Service), within the particular context and situation of each scenario
Quality of Experience and Quality of Services concepts raise complex issues. QoE is related to the perceived results of an action, as interpreted by an individual actor in a specific context. As such, QoE reporting is the subjective perception of an individual within a specific usage context considering cognitional, affective and emotional dimensions. That is to say that QoE can only be measured by users' self-reporting on their experience on criteria that ultimately the user will set himself. The meta-method needs to consider: discovering unspoken usage, needs, wants, goals & expectations of users/customers capturing users/customers creativity building customer segmentations and adapt them with agility defining a systematic process of improving reproducibility of experimentations on a large scale
The latter point requires scaling up the experiments in terms of numbers of customers, applications fields, and scope of experience without losing weak signals of usage, demand and customer expectations changes. These weak signals3 are often significant clues for the development of future offerings and experiences. One focus of the EXPERIMEDIA metamethod shall be the identification of these weak signals, as to capture them and eventually propose experiments that could provide evidence to validate their value, to whom and when.
3.2.
Challenges of Dynamic Multidisciplinary and Multicultural teams
EXPERIMEDIA consortium partners and stakeholders within target ecosystems come from diverse backgrounds. In the near future this diversity will increase through open calls and
3
By weak signals we mean reports or other data that are not within an average range, in most cases those weak signals, especially through classical data analytics are not visible. Or the future demand to be of new services, or experiences, is most often hidden behind basic statistics.
EXPERIMEDIA
possible new venues. The project partners may share a common vision for the goals but it is essential to develop an agile team that is rationally and emotionally aligned. Rational alignment is maximising the team productivity on defined executable tasks that do not require too much interactions with the other tasks, or reconsidering how one's task may affect the other. Emotional alignment is required to maximising the team flow (creativity, empathy, and ability to interact) and shall be based upon each individual personal motivation, spoken and unspoken. Although such issues may be considered as project management rather than methodology, the introduction of new partners throughout the lifetime of the project will create continuous team changes. New experiments will engage with technologists and target venue stakeholders through the methodology and therefore any approach needs to consider how to establish dialog and importantly trusted relationships for research. A key tool for multidisciplinary and multicultural teams is empathic design thinking which aims to maximise quality of experience of the end user and his group. As per the Harvard Business Review 1997 definition4: Empathic design enables companies to develop successful new products/services in their customers' own environment by identifying and addressing needs that may not be obvious, because they are tacit, unspoken or intangible. The iterative empathic design process involves five steps: customer observation in the real environment data gathering through visual, auditory and sensory cues analysis of data brainstorming development of prototypes of possible solution
In order to be able to conduct empathic design, a group needs to have unusual collaborative skills enabling creative interactions among members of a multi-disciplinary team. Empathic design is complementary to other methods and cannot replace those used for market research, multi criteria analysis, statistics and methods of simulation.
Leonard, D & Rayport, J. (1997) Spark Innovation Through Empathic Design. HBR, November 1997. http://hbr.org/1997/11/spark-innovation-through-empathic-design/ar/1
4
10
EXPERIMEDIA
4.
4.1.
Value Impact Assessment Frame (VIA)

Introduction
EXPERIMEDIA's meta-method frame is based on Value Impact Assessment (VIA). At the heart of the approach are the definition, monitoring and analysis of Business Release value measured at specific milestones throughout the execution of an experiment. Each Business Release is defined by a measurable level of some of the chosen Key Performance Indicators.
Figure 3. Value Impact Assessment Framework Overview
4.2.
Phases of the Value Impact Assessment
The overall framework for Value Impact Assessment is shown in Figure 3. The VIA framework has three phases, with each phase having specific assessment targets. Phase 1: Opportunity Assessment - a rough-cut assessment of the Business Value to scope the opportunity from a business standpoint and define a first level of assumptions about User Value. Phase 2: is the initial modelling and validation phase - a process critical to the definition of the model's limits (QoE, QoS and Business Releases). Phase 3: the industrialisation phase in which the experimental model is scaled up (this is outlined below and will be discussed in further detail after PM12).
Each phase is described in more detail in the following sections.

11
EXPERIMEDIA
4.2.1.
VIA Phase 1: Value Opportunity Assessment
Phase 1 is a high level opportunity and risk assessment as to identify with the experimenter potential benefits and risks for the experiment. The rough assessment will align all EXPERIMEDIA consortium members on the same critical issues from the business experimenter standpoint. This phase shall also identify showstoppers that may constrain an experiment. For example, there is no way of getting access to the training routine of the synchronised swimming team of Spain, at CAR, during periods prior to Olympics games. Phase 1 is really targeted to a very small number of users, so to obtain sufficient reports and insights on a defined experimental scenario. Therefore the reporting and analysis methods may be more grounded in a qualitative and ethnographic approach than any other method at this stage. Based upon these first users' feedback, a sampling of critical interactions between the user and the system may be defined as to a framework for a larger experiment aimed to test the hypothetical model.
Figure 4. Value Impact Assessment phase 1 overview
Figure 4 shows the main aspects of VIA phase 1. In terms of risks and opportunities Phase 1 will consider the following opportunities and risks:
Table 1. Opportunities and risks for value impact assessment
Opportunities The business objectives of the venue, and some of its measurement indicators from both an Economic Added Value and Return on Assets perspective (brand, image, revenue, investments, traffic, customer loyalty, etc.)
Risks Business objectives measures' uncertainty range Users goals, QoE uncertainty range, resources, skills and technological availability and costs uncertainty Legal and privacy risks
12
EXPERIMEDIA
Opportunities The assumed end user's goals and the user's critical interactions with the FMI system to be developed for this experiment, and which shall be the criteria as to measure achievement of these goals The key technological enablers and requirements that satisfy the business objectives, the user's goals, and related QoE/QoS levels. Risks
Change management issues
The outcome of phase 1 shall be a roadmap for the development and planning of the experiment based on 'opportunities and risks' of the scenario. Phase one shall also define: the first spectrum of methods, techniques and tools as to analyse the experiment the number of users, how to engage them into the experiment, and the associated risks (including privacy issues) a rough measurement system of the experiments, with targeted, time-lined levels of Business Releases, QoS, and QoE identify change management issues, showstoppers, and as to make sure experiments shall start by potential quick wins initial PIA review a detailed project management and skillset structure as to start phase 2
Phase 1 shall mainly be done through workshops, interviews and business data analysis. The phase is typically led by methodology experts within the consortium who have technology skills and knowledge of the unique characteristics of the venues where business and user value is realised. For open calls, the initial VIA will need to be conducted by the proposers, although through dedicated workshops and liaison with venue stakeholders, we expect to provide the necessary level of engagement to ensure sound value propositions are established. The experiment goals shall be linked to measurable business value and user value indicators. The business value indicators (e.g. customer loyalty improvement, traffic or occupancy increase, yielding demand or new demand creation) shall be mapped to experiment milestones called Business Releases (BR). By way of an illustration, FHW would like to experiment a new museum tour approach that provides enriched live and 3D content on demand through a smartphone within the Museum. This capability will enable a family to access a various levels of content on a specific statue of Apollo. The parents are interested in mythology, and having possibly learned some Greek at high school, would like more detailed information. In contrast, their children have not yet learned much about mythology and need access to beginner level content. The business value opportunity for FHW would be in this case: ability to show contents of high added value that cannot be easily exposed within the Museum, improve the contents' assets based on the customer feedback create loyalty with customers by proposing a subscription to content updates after they have left the museum, or propose to send them personalised updates as to make them come again 13
EXPERIMEDIA
get better insight and understanding of its customers' expectations, through measurement of the fulfilment of customers' expectations
In this example, indicators of business value could be: return on content asset business value5 traffic measurement indicators loyalty measurement indicators
Business Release 1 (BR1) would then be the first targeted achievement levels of this set of indicators for the first step of the experiment. Further BRs would be defined a points in time as a second and further steps. In summary, phase 1 has focused mainly on the Business Value indicators; QoE/QoS levels and users goals, perceptions (to be tested within a very small number of users) from the very first level of relationship of the experimenter. So the linkage between QoE, QoS and BR shall remain relatively tentative at this stage of the experiment.
4.2.2.
VIA Phase 2: Value Opportunity Validation and Macro Modelling Phase
The second phase is the engagement of lead users for small scale experimentation, on sample use cases. This phase aims to validate the opportunity assessment, and is based upon mostly empirical and qualitative observational methods and techniques. This phase shall enable: tracking, observing users' behaviour individually and collectively analysing and interpreting observations and data as to identify behavioural patterns, and critical interactions developing characteristics of those behavioural patterns per user segment isolating the top 20 and bottom 20 critical patterns, characteristics and interactions implementing systematic data capture based on interactions triggers identified and that may vary per pattern/segment iterating with users and venues as to validate and enrich sample use cases, opportunity assessment measurement levels (BR, QoE/QoS) , risks analysis so to refine and modify them
The outcome of phase 2 will be a model of:

5
data set capture modalities; data interpretation modalities and analytical tools; process and techniques on use cases samples and critical interactions behavioural patterns samples users' typology and criteria; the means to engage new users and communities
That is to say the museum has hundreds of artefacts that cannot physically be exposed, and the expert team of the museum is not even sure which of these may appeal to which audiences segments. By such an experiment, a prioritisation of artefacts can be developed. The resulting indicators would be the number and frequency of artefact interest that is not exposable. Copyright FDF and other members of the EXPERIMEDIA consortium 2012
14
EXPERIMEDIA
QoE and QoS critical criteria per behavioural pattern and users' typologies business value and business model macro design the complementary skillsets, capabilities and technologies required for phase 3 the detailed risk assessment, including PIA review, if required for phase 3
Phase 2 shall enable the assessment of the experiment roadmap, and the BR, QoE, QoS model against user experimentations and the observation results that will secure the path to phase 3.
4.2.3.
VIA Phase 3: Industrialisation Assessment Phase
Phase 3 is the industrialization assessment phase, that tests the experimental model for a large numbers of users (from a few thousand to much more). The outcome of the industrialization experimentation phase shall be a solid business model, maximizing QoE and QoS, whilst minimizing risks as to insure scalability of the experiment to big live audiences and monitor it accordingly to defined QoE and QoS levels. Besides this phase 3 shall define the scalability limits if any of the experiment. Phase 3 methods and techniques are mostly quantitative, and based on operational excellences but also shall include a built-in, closed feedback loop with user panels that will track services capabilities limits, areas for improvement, and weak signals. The feedback mechanisms shall be as automated as possible at this stage and linked as directly as possible to the quality assurance and monitoring processes and mechanisms of EXPERIMEDIA facility.
4.3.
VIA Review Board Processes
Of course, each phase of the VIA framework can be iterated as many times as required prior to entering the next phase. A formal opportunities/risks review will be undertaken at each phase to decide if to proceed to the next phase of the experiment or if an iteration is required, and advise experimenters in areas that further investigation or expertise. The review board shall be directed by the EXPERIMEDIA's Experiment Manager with all partners involved in the experiment and invited experts used on demand as required. The VIA review boards shall be formal meetings, and all presentations sent a week prior to the review board. Templates shall be developed for the review boards, and include a decision summary dashboard of the review board, with a named, time-lined action plan. Ethical, methodological and architectural task leaders shall arbitrate decisions in situations where there are disagreements at the review board.
15
EXPERIMEDIA
5. Methodological Approach towards Legal and Ethical Requirements

5.1. Introduction
The social and networked media experiments targeted at the EXPERIMEDIA facility will have to take into account a number of ethical concerns, mainly relating to the protection of personal data and the privacy of the users involved in such experiments. To facilitate a more ethical methodology for carrying out these experiments, this chapter will discuss how ethical and privacy concerns need to be addressed throughout the experiment lifecycle. An important note to be made is that the results presented in this chapter will work in close conjunction with the work to be performed throughout the course of the project, taking into account the legal and ethical requirements. For one, ethics oversight within the EXPERIMEDIA consortium will mainly be addressed by the Data Protection Coordinator, who will liaise with national Data Protection Authorities and internal ethics committees of all partners. To the broader community outside the consortium, the Ethics Advisory Board will be led by one of its Members and will be composed of experts in the fields. In order to ensure high quality of the work conducted within the EXPERIMEDIA project, the compliance with the current legal framework applicable to that work as well as to the social and networked media experiments has been defined as sine qua non. At the level of EU legislation, three directives can already be identified as being important instruments for this project. First, Directive 95/46/EC on the protection of individuals with regard to the processing of personal data and on the free movement of such data - also referenced to as the Data Protection Directive - provides the general legal framework applicable to the processing of personal data. Second, Directive 2000/31/EC on certain legal aspects of information society services, in particular electronic commerce, in the Internal Market - the Directive on electronic commerce - provides a number of basic definitions on information society services, including the liability of intermediary service providers. Third, Directive 2010/13/EU concerning the provision of audiovisual media services "the Audio-visual Media Services Directive" creates a level playing field for emerging audio-visual media and provides a safeguard for media pluralism. Given the importance of the general legal framework set by these three main EU-wide instruments and their subsequent national implementations by the different EU Member States, the directives enumerated here will be further analysed in the frame EXPERIMEDIA's legal analysis. The analysis performed will lead to the formulation of legal requirements and recommendations that will aid overall legal and ethical compliance of the research work performed in EXPERIMEDIA. Given the close relationship between law and ethics, a number of ethical issues will - in part already be referenced in the currently applicable legal framework. As already mentioned, the main ethical concern that can be raised with regards to the research that will be performed during the course of EXPERIMEDIA and the social and networked media experiments is related to the processing of the personal data of the users involved and the protection of their privacy. To perform the research as planned, it is important that the behaviour of users in online and real-world communities is understood, tracked and modelled. In
16
EXPERIMEDIA
doing so, certain types of personal data may be captured and processed, which in turn raises ethical and legal concerns relating to the privacy of the citizens resulting from the processing of his personal data. In this context, it is important that it is clearly understood what the potential impact of the social and networked media experiments is on the privacy of the citizen. The main instrument for addressing such privacy concerns is by conducting a Privacy Impact Assessment.
5.2. 5.2.1.
Privacy Impact Assessment (PIA) From a Privacy Impact Assessment on RFID
Privacy Impact Assessments have been carried out in relation to systems and applications that present privacy aspects and interest, but the concept of a Privacy Impact Assessment (PIA) has become broadly known via the discussions regarding the use of Radio-Frequency Identification (RFID) technology. As this technology makes it possible to track and possibly even identify users, the use of RFID poses a number of concerns regarding their privacy. However, given its economic potential, the use of RFID is steadily becoming an integral part of everyday life. It is therefore why different legislators, such as the European Commission, and advisory bodies, such as the Article 29 Working Party, have voiced their opinion on this matter. Already in 2007, the European Commission saw the potential benefits of RFID and called for solutions to remedy the potential negative effects of this technology.6 The European Data Protection Supervisor, for one, coined the idea of performing an assessment of the expected impact of specific RFID applications to the privacy of their users as each application may pose a different level of threat to the users privacy, as well as that of implementing security and privacyby-design principles in deploying RFID applications.7 The Commission subsequently called for a framework developed at Community level for conducting privacy and data protection impact assessments [that] will ensure that the provisions of this Recommendation are followed coherently across Member States.8 Such impact assessment would be able to provide the information required for appropriate protective measures and would guide Member States on the design and operation of RFID applications in a lawful, ethical and socially and politically acceptable way, respecting the right to privacy and ensuring protection of personal data.9 This demonstrates a clear preference of the Commission to implement selfregulating measures instead of proposing additional legislation to deal with this matter. As such, the Commission asked the Member States to ensure that industry, together with civil society stakeholders, would develop a framework for privacy impact assessments to provide an answer to the legal and ethical concerns regarding the implementation of privacy-threatening RFID applications. A first draft of such framework was delivered to the Article 29 Working
Communication from the Commission to the European Parliament, the Council, The European Economic and Social and the Committee of the Regions, "Radio Frequency Identification (RFID) in Europe: steps towards a policy framework", COM(2007) 96 final. 7 Opinion of the European Data Protection Supervisor on the communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions on Radio Frequency Identification (RFID) in Europe: steps towards a policy framework COM(2007) 96, OJ C101 of 23 April 2008, 1-12. 8 Commission Recommendation of 12 May 2009 on the implementation of privacy and data protection principles in applications supported by radio-frequency identification, C(2009) 3200 final, 4. 9 Ibid., 5.
6
17
EXPERIMEDIA
Party in 2010 and discussed in an opinion of this advisory board.10 In its analysis, the Working Party identified the three main goals of the Privacy Impact Assessment: 1) Promote Privacy by Design by helping data controllers to address privacy and data protection before a product or service is deployed. 2) Help data controllers to address privacy and data protection risks in a comprehensive manner. 3) Help both data controllers and data protection authorities to gain more insight into the privacy and data protection aspects of RFID Applications.11 The first PIA framework proposed to the Article 29 Working Party included four risk levels that corresponded to the level of detail required in the PIA. The Article 29 Working Party, however, was not satisfied with the level of risk assessment in this proposed PIA framework and therefore decided not to endorse the proposal. A similar position was adopted by the European Network and Information Security Agency (ENISA).12 This agency proposed to include a more comprehensive and recognized methodological basis for the PIA, which makes use of existing frameworks, standards and best practices as well as a clearer risk assessment.13 A revised framework was proposed in 2011 and was again analysed by the Article 29 Working Party.14 The new framework introduces a dual procedure. First, the pre-assessment phase qualifies the level of risk of the RFID application, leading to the need for either no further analysis, a small-scale PIA or a full-scale PIA. Second, in the risk assessment phase the actual PIA is performed. Main focal points here are the characterization of the application, the identification of the risks to personal data, the identification and recommendation of controls in response to such risks and the documentation of the results of the PIA. Also a consultation of stakeholders is included in the PIA. Given these improvements, the Article 29 Working Party endorsed the revised PIA framework and called for its implementation. The PIA framework was officially signed on 6 April 2011.15
5.2.2.
to a general Privacy and Data Protection Assessment?
While the previously analysed PIA framework is specifically aimed at assessing the privacy risks and threats of RFID applications, it does provide a number of general ideas that could serve as a basis for a wider PIA framework, for application to other situations such as the work performed under EXPERIMEDIA and the social and networked media experiments. Also the Commission is now calling for
an obligation for data controllers to carry out a data protection impact assessment in specific cases, for instance, when sensitive data are being processed, or when the type of processing
Article 29 Working Party, Opinion 5/2010 on the Industry Proposal for a Privacy and Data Protection Impact Assessment Framework for RFID Applications, WP 175. 11 Ibid., 5-6. 12 ENISA, Position on the Industry Proposal for a Privacy and Data Protection Impact Assessment Framework for RFID Applications of March 31, 2010, enisa.europa.eu. 13 Ibid., 6-7. 14 Article 29 Working Party, Opinion 9/2011 on the revised Industry Proposal for a Privacy and Data Protection Impact Assessment Framework for RFID Applications, WP 180. 15 ec.europa.eu/information_society/policy/rfid/documents/rfidpiapressrelease.pdf.
10
18
EXPERIMEDIA
otherwise involves specific risks, in particular when using specific technologies, mechanisms or procedures, including profiling or video surveillance.16
Guidelines to perform a more general PIA can already be found in several countries, such as Australia, Canada, New Zealand, the UK and the US. Also the International Organization for Standardization (ISO) has performed work towards a more generally applicable PIA.17 While a more general PIA framework could certainly find support, the question is whether such a framework should be a self-regulatory one - like the RFID PIA framework - or an obligation imposed by the government. At the moment, only Canada and the US seem to have introduced a level of mandatory PIAs.18 Although it could be argued that mandatory PIAs could help to raise the level of transparency and accountability in personal data processing by organizations, it could also be argued that mandatory PIAs would increase administrative burdens and that their potential would be diminished if the PIA is reduced to another act of regulatory compliance.19 To mitigate the potential negative impact of a mandatory PIA, a balanced approach is therefore of utmost importance. Although the debate on whether a more general PIA should be made mandatory or not is still on going, the previously mentioned 2010 Commission Communication does indicate that the EU is more akin to introducing a general obligation to conduct a PIA. Therefore, it would be recommended to develop a common methodology for conducting a PIA within the EXPERIMEDIA framework for use in relation to the social and networked media experiments. As such, this PIA methodology could help to identify and remedy, in a timely fashion, the privacy concerns that may arise during the course of such experiments. With privacy concerns being the main ethical and legal issue identified in EXPERIMEDIA, the PIA methodology could aid in addressing that issue and should therefore be seen as an integral part of the general EXPERIMEDIA methodology.
5.3.
PIA methodology development
While the RFID PIA already provides a number of general notions that could be translated into a more broadly aimed PIA (or a more specifically aimed PIA, with regards to the EXPERIMEDIA experiments) its very narrow focus on RFID applications bar this methodology from serving as a PIA for different applications as such. Therefore, the EXPERIMEDIA PIA methodology will draw from a greater pool of existing PIA initiatives, in order to provide a more comprehensive view on this issue. Notably, the EU itself has decided to follow a similar approach: the currently on going PIAF project analyses a number of existing PIAs in order to determine the best elements from these examples.20
Communication from the Commission to the European Parliament, the Council, The European Economic and Social and the Committee of the Regions, " A comprehensive approach on personal data protection in the European Union", COM(2010) 609 final, 12. 17 See: ISO 22307:2008: Financial servicesPrivacy impact assessment. While the standard is aimed at financial services, it also provides some more general requirements for PIAs. 18 D. Wright, "Should Privacy Impact Assessments Be Mandatory?", Communications of the ACM, August 2011, Vol. 54, no. 8, 127. 19 Ibid., 128. 20 www.piafproject.eu.
16
19
EXPERIMEDIA
While as many definitions as methodologies exist for the concept of the PIA, this analysis will follow the definition that a PIA is a
methodology for assessing the impacts on privacy of a project, policy, programme, service, product or other initiative which involves the processing of personal information and, in consultation with stakeholders, for taking remedial actions as necessary in order to avoid or minimise negative impacts.21
Important to note is that a PIA is, by definition, a prospective study and should be performed at the earliest possible stage of a proposal as part of the general risk management strategy, in order to ensure that sufficient adaptations can still be timely implemented to mitigate privacy risks. Therefore, the PIA must be distinguished from the compliance check and the audit, as these are generally performed at a later stage.22 A PIA should also address broader legal and ethical issues, while compliance checks and auditing generally focus on very specific issues. It should also take into account different types of privacy, such as the privacy of the person, his behaviour, his communications, his location, etc.23 In order to address and uncover a broader range of ethical and legal issues, the PIA must have a wide scope in order to be successful. A limited scope would allow for just briefly touching upon the major issues, thus potentially overlooking other underlying issues.24 An important element here is the level of guidance offered to the entity responsible for carrying out the PIA. A limited questionnaire will be less able to uncover the specific privacy concerns than a more detailed analysis which includes a mapping of the personal data information flows. As such, the PIA guidance documents should also include a threshold assessment, to ensure whether a PIA is really needed.25 Additionally, guidance documents could also include potential mitigation strategies for the risks defined in the PIA.26 Such strategies should, however, be used with caution as every project is unique and may therefore require a different approach. Possible mitigation strategies included in the PIA guidance documentation should therefore only serve as non-exhaustive examples. Another important element that should be sufficiently present in the PIA is the documentation of information flows.27 By clearly indicating which information is collected, by whom, for which purposes, for how long, etc. the PIA can provide a clear overview of the potential risks of such information flow.
D. Wright, K. Wadhwa, P De Hert, D. Kloza, "PIAF D1: A Privacy Impact Assessment Framework for data protection and privacy rights", piafproject.eu, 14. 22 Linden Consulting, "Privacy Impact Assessments: International Study of their Application and Effects", Prepared for the Information Commissioners Office (UK), October 2007, 1-2. 23 D. Wright, K. Wadhwa, P De Hert, D. Kloza, "PIAF D1: A Privacy Impact Assessment Framework for data protection and privacy rights", piafproject.eu, 189. 24 Linden Consulting, "Privacy Impact Assessments: International Study of their Application and Effects", Prepared for the Information Commissioners Office (UK), October 2007, 16-17. 25 D. Wright, K. Wadhwa, P De Hert, D. Kloza, "PIAF D1: A Privacy Impact Assessment Framework for data protection and privacy rights", piafproject.eu, 191. 26 Office of the Victorian Privacy Commissioner, "Accompanying Guide: A guide to completing Parts 3 to 5 of your Privacy Impact Assessment Report", 2009. 27 Ibid., 189.
21
20
EXPERIMEDIA
While a PIA is mainly an early indicator of potential privacy risks, it must also be seen as an ongoing process.28 As a project or an experiment progresses, more and more details become apparent, which in turn may lead to the identification of new privacy risks. An early-stage PIA will therefore identify the main privacy concerns, while on-going re-assessment and evaluation is needed to keep the PIA up-to-date. Additionally, PIAs could also lead to a higher level of transparency and accountability by going beyond the scope of being a merely internal self-assessment exercise. An important element here is submitting the PIA report to an external ethical and legal board, or even to a national privacy regulator.29 This also includes consulting with external stakeholders that are directly affected by the proposal. Due to that position of being directly affected, these stakeholders could provide a deeper or different insight into the proposal, possibly uncovering privacy concerns that the organization conducting the PIA was previously unaware of. As such, external stakeholders could also aid in developing remedies for such privacy concerns. They can also provide access to a wider range of knowledge, as the entities entrusted with performing the PIA may lack certain knowledge in fields like privacy legislation, security requirements, etc. Within EXPERIMEDIA, the Data Protection Coordinator will fulfil the role of securing external validation through its liaising with external ethics committees and the project's Ethics Advisory Board. With regards to the outside community, it should also be noted that publishing the PIA report can lead to greater transparency regarding the conduct of PIAs and may thus help to enhance the overall perception and legitimacy of PIAs. As many organizations wish to keep their PIA reports private, there are on a global scale only limited PIA reports open for public consultation.30 This limited availability of PIA reports makes it a difficult exercise to perform a comprehensive study of existing PIA standards and practices in order to determine the best practices in this field for further evaluation and development. The diversity of PIA approaches around the world can also serve as a reminder that a PIA is not always a very straightforward process. While national or local authorities can provide general requirements and guidelines for PIAs, each PIA report should be tailored to the needs and goals of the specific entity performing the PIA.
5.4.
EXPERIMEDIA PIA
The PIA methodology explained here must be understood as providing a number of general guidelines that indicate the most important elements of an adequate PIA as well a number of best practices identified in the divergent field of PIAs. As a result, this methodology should by no means be interpreted as providing an exhaustive and definitive list of steps that need to be taken in the process of conducting a PIA. Persons involved in conducting a PIA of their planned projects - or experiments in the EXPERIMEDIA context - are therefore strongly encouraged to use the discretion left to them to tailor the PIA to the specific needs and goals of their situation. The methodology described here will consist out of a number of different phases: preparation, pre-assessment, risk assessment, documentation and reporting, and review. The process of the PIA methodology can be summarized in the diagram below:
Linden Consulting, "Privacy Impact Assessments: International Study of their Application and Effects", Prepared for the Information Commissioners Office (UK), October 2007, 21. 29 Ibid., 22-23. 30 Ibid., 25-26. D. Wright, K. Wadhwa, P De Hert, D. Kloza, "PIAF D1: A Privacy Impact Assessment Framework for data protection and privacy rights", piafproject.eu, 17-18.
28
21
EXPERIMEDIA
Figure 5. Privacy Impact Assessment methodology process
The PIA process aims to uncover potential privacy risks that may arise from the project - or experiment in the EXPERIMEDIA context - and will at the same time propose mitigation strategies to remedy those risks. By performing this exercise at the earliest possible stages of the project, the PIA should allow for sufficient time to implement the necessary amendments and safeguards to ensure that privacy is taken into account by design, rather than being added at the end of the project development. As such, the PIA is a most valuable tool in addressing the ethical concerns that may arise within the EXPERIMEDIA framework and the experiments conducted therein, as privacy and data protection were already indicated as being the main ethical concerns in this context. The EXPERIMEDIA will serve as the basis for the legal and ethical analysis in EXPERIMEDIA.
5.4.1.
PIA Phase 1: Preparation
Before commencing the actual work on the PIA, the organization performing the PIA will need to go through an internal procedure to lay down the basic rules for the PIA. First, the PIA needs to be scheduled early enough in the project to allow sufficient time for the changes needed to mitigate privacy risks. The organization also needs to determine the person(s) responsible for conducting the PIA. This may be a specifically appointed privacy officer or any other person involved. Important here is that the PIA, due to its broad nature, should include information stemming from different perspectives (e.g. legal, technical, managerial, etc.) and that the final report should be approved at the senior or executive level. When an organization does not have direct access to sufficient resources to conduct a comprehensive PIA, external consultants and experts should be contacted. In any case, external
22
EXPERIMEDIA
contacts with consultants and stakeholders are strongly encouraged to receive different insights in the potential privacy risks and the mitigation thereof. Also national privacy authorities are an important entity to involve in the PIA. Within the EXPERIMEDIA framework, the Data Protection Coordinator can act as liaison between the experiment applicants and the outside community. As the PIA should be considered as an on-going process, the organization should also plan for a timetable in which a review of the PIA is planned, or for events that would trigger the need for a revision of an earlier PIA. Also in the review, contact with external consultants and stakeholders is of importance to ensure the involvement of a larger community and different disciplines.
5.4.2.
PIA Phase 2: Pre-assessment
After the basic rules of PIA conduct have been determined by the organization performing the PIA, the pre-assessment phase provides a number of steps that will determine the further procedure of the PIA. It is in this phase that it is determined whether a PIA is needed or not and, if needed, whether the PIA should be small-scale or full-scale. The pre-assessment phase is therefore an important step in the PIA process as an unnecessary or unnecessarily broad PIA could be costly to an organization in terms of time and resources. An important first step in this process is to gather all relevant information that can provide guidance in determining the scope of the PIA to be applied. First, this means that information on the project and its scope must be collected. As the PIA should be performed as early in the project as possible, it may be that there is not much information available yet, for instance with regards to the precise scope and goals of the experiments. Even when short on concrete information, all available documents should be gathered and project officials should be consulted.31 A preliminary outline of the project should be able to aid in determining the scope of the PIA, if applicable. Also a list of potential stakeholders should be compiled. Not only will such list serve in the later stages of the PIA, it can also help in establishing contact points for expertise not available within the project or help in determining the general scope and impact of the project. This has already been realised via the composition of the Ethics Advisory Board (see Deliverable D5.1.1 EAB and DPB operating procedures32). Also publicly available PIA reports of other similar projects may be collected to provide some general ideas of how the PIA works. Given the many differences between currently available PIAs, such reports should only serve for exemplary purposes. Existing consultations with relevant stakeholders and experts could be also considered to provide a basis of information that may be of use during the course of the PIA. Once all relevant information has been gathered, it can be put to use in determining whether a PIA is needed and, if so, to which extent. In general, the following questions need to be raised: 1) Does the project include the processing of personal data33? 2) Does the project include activities that may link specific data to a natural person? These questions can be summarized in the following table:
ICO, "Privacy Impact Assessment Handbook V2", ico.gov.uk, 2009, 28. http://www.scribd.com/doc/79825565/D5-1-1-EAB-and-DPB-Operating-Procedures-v1-0 33 Understood as determined in article 2(a) of the Data Protection Directive.
31 32
23
EXPERIMEDIA
Table 2. Privacy Impact Assessment level decision table
Question
Answer Yes No Go to question 2 No PIA required
1. Is personal data being processed? 2. Is data linked to a natural person?
Full-scale PIA needed Small-scale PIA needed
As the table indicates, there is a basic level at which no PIA action needs to be undertaken. This is the case when no personal data is processed within the project and when no specific data is linked to a natural person. At a higher level, it is clear that a PIA is needed, which leaves the determination of the scale of the PIA. When specific data can be linked to a natural person, but no personal data is processed, a small-scale PIA should suffice. When personal data is processed within the project, a full-scale PIA would be recommended. Note, however, that this table represents only the most basic level at which the need for a PIA can be determined. The UKs Information Commissioners Office, for instance, has drafted guidelines with a much more extensive decision table, which may provide further guidance in determining the level of the PIA required.34 These guidelines also reference the need for a data protection compliance check when personal data is handled, whereby personal data is understood as any information relating to an identified or identifiable natural person, with a person being identifiable if he can be directly or indirectly identified by reference to identification numbers or other specific factors relating to his physical, physiological, mental, economic, cultural or social identity. Additionally, a privacy law compliance check is needed when any of the envisioned activities are subject to the existing privacy laws or contrary to good privacy practices.35 Another approach in determining whether a PIA is needed can be found at the Office of the Victorian Privacy Commissioner.36 This agency has developed a simple questionnaire that can lead an organization to determine whether the threshold for a PIA was reached or not. The 17 questions included here refer to the establishment of public registers; the use, collection and disclosure of personal data; security and confidentiality of personal data; the use and creation of identification systems; the exchange of personal data; and surveillance measures. If any of the questions on the list apply to the proposed project, a PIA is needed. The Office of the Victorian Privacy Commissioner does, however, not distinguish between a small-scale and full-scale PIA. While such basic questionnaire can indeed be helpful in determining a minimum threshold for conducting a PIA, a static approach like this does not leave much room for deviation and tailoring to the needs of individual projects. Therefore, the approach followed by the EU in the RFID PIA can be recommended, as it includes a PIA threshold as well as a distinction between a lower-level small-scale PIA and a higher-level full-scale PIA.
ICO, "Privacy Impact Assessment Handbook V2", ico.gov.uk, 2009, 54-60. While the EU RFID PIA does not include the intermediary levels of the data protection compliance check and the privacy compliance check, it should be noted that data protection and privacy laws must be complied with at all times and that a timely check for such compliance would therefore be recommended. 36 Office of the Victorian Privacy Commissioner, "Privacy Impact Assessments: A guide for the Victorian public sector", 2009, 22.
34 35
24
EXPERIMEDIA
The general recommendation to be made here is to start the process using the questions in Table 2 listed above. The basic questions there will provide an initial assessment of the need for a PIA and the potential scope thereof. The ICO or Victorian Privacy Commissioner guidelines can subsequently provide additional guidance when the initial assessment has not provided sufficient clarity on the matter.
5.4.3.
PIA Phase 3: Risk Assessment
The risk assessment phase is the most important phase of the process, as here the actual PIA is conducted. In this phase, the proposal will be analysed to identify potential privacy risks and to document how these risks could be mitigated. Early identification and mitigation of potential privacy risks can be done by ensuring that privacy is respected within the project from the design up, instead of added at the final stages. While this analysis may contain a certain degree of compliance checking as well, the PIA remains above all a risk assessment tool. Note that the actual procedure for a full-scale PIA and a small-scale PIA is in essence the same. The main difference is that when a full-scale PIA is required, a higher level of detail with regards to scope, inquiry and reporting is expected. A small-scale PIA therefore requires fewer formalities, less resources and a less exhaustive analysis. As a first step, the project must be adequately described. This description needs to give a full and comprehensive overview of the project itself, its environment and its system boundaries.37 Also the information flows envisioned within the project need to be described in as much detail as possible. A diagram showing the flow of information between the parties involved within the project (for instance Figure 6 below) may aid in providing a clearer and visual picture of these data flows. This element is also important for determining the potential links that can be made between different data. These data links are highly important from a data protection point of view as data link ability can lead to the identification of the data subject. The description made here can make use of the information gathered in the pre-assessment phase. As such, the description should include the goals of the project, potential stakeholders, etc. As a general rule, the project description must include the name and location of the project coordinator, the purposes of the project, a description of the technology that will be used and of the geographical scope where the project will be conducted, an overview of the types of individuals that may be impacted by the project, a list of the data types that will be processed, whether and which (sensitive) personal data will be processed, the duration of storage of the data, a description and/or visualization of the data flows of the project, potential transfers and recipients of personal data, and whether the data will be transferred outside of the EEA.
INFSO, "Privacy and Data Protection Impact Assessment Framework for RFID Applications", INFSO-201100068, 12 January 2011, 9.
37
25
EXPERIMEDIA
Figure 6. Sample data flow diagram38
Second, the potential privacy risks that may arise from the project must be identified and listed. Important here is to understand the concept of risk as being the likelihood of a certain threat occurring and the potential impact thereof. When identifying risks in this phase of the PIA process, the search will therefore be on potential threats to the privacy of the natural person, the likelihood of these threats materializing and their potential impact in such event. Threats can be defined as being the possibility of a certain goal not being reached, or of the opposite event occurring. As such, threats can be deducted from those goals, also addressed as targets. If a goal or target is set, threats are therefore the situations in which the goal or target will not be reached. One important source for defining privacy targets is the EU Data Protection Directive. In general, one can find nine targets listed in this directive 39 as reproduced in Figure 7 below. By understanding these targets, the potential threats to them can be systematically identified. The threats identified by the EU RFID PIA40 include, for example, the absence of a purpose for the processing of personal data, processing exceeding that purpose, unclear or non-transparent information given, lack of consent, secret data processing, insufficient measures for data security and confidentiality, non-compliance with the data subjects rights, etc. While these threats are solely derived from the EU Data Protection Directive, a general look at the notion of privacy and its ethical and legal connotations could lead to a more extensive field of potential threats to be identified. For instance, the Data Protection Directive does not directly refer to the issue of private electronic communications, which could possibly also be threatened in the proposed project. Potential stakeholders and external consultants may be involved in this phase as well to ensure a broader knowledgebase and mind-set when identifying potential threats.
Office of the Victorian Privacy Commissioner, "Privacy Impact Assessments: A guide for the Victorian public sector", 2009, 17. 39 INFSO, "Privacy and Data Protection Impact Assessment Framework for RFID Applications", INFSO-201100068, 12 January 2011, 13. 40 Ibid., 14-16.
38
26
EXPERIMEDIA
Figure 7. EU data protection privacy targets41
Apart from identifying the potential threats to the privacy, a PIA must also quantify those risks. This means that the likelihood of those threats materializing must be assessed in order to determine the urgency for mitigation of such threats. Not only does the likelihood of a threat materializing need to be assessed, but also the potential magnitude of such event. As such, potential threats can be classified - taking into account their likelihood and magnitude according to a risk level ranging from low to high. Third, current and proposed controls to mitigate identified risks must be documented. The goal here is to minimize, mitigate or eliminate the identified privacy risks. Such controls can be adopted at a technical level, for instance by using encryption methods and strong authentication mechanisms, or at a more operational level for example by implementing specific policies for preserving privacy. Certain controls could be aimed at preventing a threat from materializing, while others are aimed at remedying threats that already materialized. When adopting a risk mitigation strategy, it is important to follow the risk level assessment performed earlier, to ensure that all risk levels are brought to an acceptable level. With regards to the threats to the Data Protection Directive, it can be suggested to ensure compliance with the legal framework set out by that directive to ensure the mitigation of potential threats arising from non-compliance. Other mitigation measures can be found in adopting governance practices, that need to ensure good
Source: INFSO, "Privacy and Data Protection Impact Assessment Framework for RFID Applications", INFSO2011-00068, 12 January 2011, 13.
41
27
EXPERIMEDIA
management practices within the project; providing individual access and control to the data subject and informing him about the possibility to withdraw his consent; adopting adequate system security measures including the use of audit controls; and including accountability measures. As part of the risk management strategy and general compliance with EU data protection legislation, it can be recommended to include a proportionality test to assess whether the mentioned processing of personal data is truly necessary for the achievement of the project's goals or whether there would be a less privacy intrusive measure that could lead to the same results. In essence, the interests at stake - privacy of the data subject versus project goals at large - would need to be weighed off against each other and be balanced. Subsequently, the resolution, being the results of the analysis, needs to be documented and reported, which will be discussed as a next phase of the process.
5.4.4.
PIA Phase 4: Documentation and Reporting
The results of the risk assessment and mitigation strategy, generally addressed as the resolution, have to be documented in a report. This report also must include additional remarks on the risks, potential mitigation strategies and residual risks. If the PIA assessment has made clear that the project as it currently stands does not pose any significant privacy risks that cannot be appropriately mitigated, the project can be approved for further development. However, when no appropriate mitigation strategy can be developed or when a number of residual risks remain, the project will require further corrective measures and a subsequent review of the PIA before further development can resume. As already stated in the preparatory phase, the organization conducting the PIA will have to assign a senior or executive officer to sign the resolution. This is important to ensure that the PIA report has sufficient internal legitimacy and to demonstrate that it has passed through different levels of the organization rather than being a one-man effort. Another reason to have the report pass through senior or executive levels is because the analysis performed in the PIA may include sensitive information subject to corporate security and confidentiality measures as this report needs to describe the project and document the PIA process. When such sensitive and confidential information is present, it may be relocated to confidential appendices to the report.42 Such practice should, however, be limited to ensure maximum publicity and transparency of the report. As such, a valid justification is needed for removing certain information from the public report. In order to have an adequate PIA report it must include a comprehensive project description, a full risk analysis, a justification of the potential privacy risks with analysis of potential lessintrusive alternatives and a risk mitigation strategy including privacy-by-design features. Where applicable, the sources consulted throughout the PIA process should be documented as well. These sources may include stakeholder consultation, contacting external experts and consultants, etc.
42
ICO, "Privacy Impact Assessment Handbook V2", ico.gov.uk, 2009, 40.
28
EXPERIMEDIA
According to the PIAF project, a few useful criteria for the PIA report are43: 1) Clarify whether the PIA was initiated early enough so that there was still time to influence the outcome 2) State who conducted the PIA 3) Include a description of the project to be assessed, its purpose and any relevant contextual information 4) Map the information flows (i.e., how information is to be collected, used, stored, secured and distributed and to whom and how long the data is to be retained) 5) Check the projects compliance against relevant legislation 6) Identify the risks to or impacts on privacy 7) Identify solutions or options for avoiding or mitigating the risks 8) Make recommendations 9) Be published on the organisations website and be easily found there or, if the PIA report is not published (even in a redacted form), there should be an explanation as to why it has not been published 10) Identify what consultation with which stakeholders was undertaken.
5.4.5.
PIA Phase 5: Review
The last element of the PIA process is the review phase. This phase contains a number of different elements. First, a PIA report should be made public. One important reason hereto is that the PIA process may have involved external stakeholders and consultants, and publication of the report would provide them with the opportunity to review how their contributions were documented. The report should also be made available to national privacy authorities, so that they can see whether the PIA was performed according to expectations and whether any privacy and data protection risks remain applicable. The publication of the report is important for several purposes, mainly to raise accountability of the organization performing the PIA, to lead in to a postimplementation review of the proposed risk management strategy, to allow for auditing, to provide input into future iterations of the PIA, and to provide background information for people conducting PIAs in the future.44 Apart from giving these parties access to the report, they may also be actively involved in providing feedback to the PIA. This way, other insights can be provided even after the finalization of the report. As a PIA is an on-going process, the report may be updated as new insights become available. Also for internal purposes, the report may have to be reviewed. The main reason here is that it needs to be assessed whether the risk management strategy proposed in the PIA report is properly adopted and implemented. This again corresponds to the idea that a PIA is an on-going process and that existing insights and resolutions may have to be updated. In any case, it is strongly advised to schedule for regular intervals at which the PIA report should be updated, in order to ensure
As presented by K. Wadhwa, "A review of PIA reports in 5 countries", presented at the PIAF workshop, 12 October 2011, Brussels, http://piafproject.eu/ref/A+Review+of+PIA+Reports+in+Five+Countries+Final.pdf, 4. 44 ICO, "Privacy Impact Assessment Handbook V2", ico.gov.uk, 2009, 41.
43
29
EXPERIMEDIA
that the report keeps showing the latest developments in the risk identification and analysis and the mitigation strategy building. Alternatively, a number of potential events that would trigger the need for a review of the PIA could be established.
5.5.
PIA Outcome
While the bulk of the work in the PIA process will culminate in the PIA report, the fifth phase of the process clearly indicates that the PIA will remain an on-going process. The report will therefore have to be updated throughout the development process of the project to ensure that all identified risks are properly mitigated and that no residual risks remain unaddressed. Additionally, as the project takes more shape during its development process, the PIA needs to ensure that new risks that arise are timely and adequately identified and mitigated. For external validity of the process, the EXPERIMEDIA infrastructure has foreseen a Data Protection Coordinator and an Ethics Advisory Board. These entities act as the main ethics watchdogs of the EXPERIMEDIA project and will in these functions receive and review the PIA reports submitted by the experiment proposal applicants. It is also in these functions that outreach will be made to the outside community to ensure that the reports are presented to the larger society and that relevant stakeholders and consultants are given access to the PIA work performed here. However, while the PIA is an important instrument in assessing and addressing the general ethical concerns that could present themselves within the course of the work planned in EXPERIMEDIA and the social and networked experiments, there are also limits to what a PIA can do. Although the PIA mainly addresses privacy concerns and thereby also in a way addresses the legal concerns stemming from privacy and data protection legislation, it is not a legal compliance tool on its own. Therefore, to ensure compliance with the general legal framework applicable to the EXPERIMEDIA framework and the social and networked experiments, an additional analysis of compliance with that legal framework needs to be conducted and it will be realised in D5.1.2 First legal and ethical framework for the deployment of EXPERIMEDIA testbeds and experiments. This legal framework spans a broader scope than the limited privacy and data protection legislation referenced to in the PIA. While the PIA can be regarded as an important first step in a project to ensure that ethical and legal concerns are properly addressed, further guidance in the form of legal requirements is needed for a broader spectrum of legislation. This will be addressed in Work Package 51.
30
EXPERIMEDIA
6. Conclusion
This document has described the first version of guidelines for conducting experiments at EXPERIMEDIA facilities. The document has outlined the primary challenges and principles for the design of social and networked media experiments using Future Media Internet (FMI) technologies. Considering the types of experiments to be conducted, there is a need for a balanced and iterative approach that considers the interplay between FMI technologies, venue ecosystems and users participating. Special attention is necessary for the engagement of users during assessment of experience and privacy impact. The dynamic, multidisciplinary and multicultural nature of teams within the project raises additional challenges that need to be addressed both rationally and emotionally. This initial document provides a meta-method frame for experiments based on Value Impact Assessment (VIA) and Privacy Impact Assessment (PIA). VIA focuses on defining, measuring and assessing business release value at specific milestones during the lifetime of an experiment. The business release value is defined in terms of Key Performance Indicators derived from Quality of Service (QoS) and Quality of Experience (QoE) metrics. VIA is organised into three phases, each designed to incrementally move towards industrialisation and large scale trials. PIA focuses the formal assessment of privacy risks and ensures that experiments conducted using the EXPERIMEDIA facility take into account ethical concerns, mainly relating to the protection of personal data and the privacy of the users involved in such experiments. PIA has five different phases: preparation, pre-assessment, risk assessment, documentation and reporting, and review. The outcome of the PIA assessment will report on identified risks and mitigation strategies to be implemented to ensure that no residual risks remain unaddressed. The meta-method frame will now be elaborated through use by the driving experiments of EXPERIMEDIA at Schladming Ski Resort, Multi-Sport High Performance Centre of Catalonia (CAR) and Foundation of the Hellenic World (FHW). Specific methods will be defined for the desired experience patterns within each of the experimental scenarios. These patterns of experience will provide methodology templates for future experiments to be conducted at the EXPERIMEDIA facility.
31

D2.1.1 First EXPERIMEDIA Methodology v1.02

Încărcat de

Informații document

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

D2.1.1 First EXPERIMEDIA Methodology v1.02

Încărcat de

Drepturi de autor:

Formate disponibile

D2.1.

1 First EXPERIMEDIA Methodology

Copyright FDF and other members of the EXPERIMEDIA consortium 2012

Methodological Principles and Challenges ..................................................................................... 7

Value Impact Assessment Frame (VIA) ....................................................................................... 11

Copyright FDF and other members of the EXPERIMEDIA consortium 2012

Copyright FDF and other members of the EXPERIMEDIA consortium 2012

Copyright FDF and other members of the EXPERIMEDIA consortium 2012

Figure 1. The relationship of methodology to EXPERIMEDIA activities

Copyright FDF and other members of the EXPERIMEDIA consortium 2012

Copyright FDF and other members of the EXPERIMEDIA consortium 2012

3. Methodological Principles and Challenges

Copyright FDF and other members of the EXPERIMEDIA consortium 2012

Challenges for Evaluation of FMI Systems with Users

Challenges of Dynamic Multidisciplinary and Multicultural teams

Copyright FDF and other members of the EXPERIMEDIA consortium 2012

Copyright FDF and other members of the EXPERIMEDIA consortium 2012

Value Impact Assessment Frame (VIA)

Figure 3. Value Impact Assessment Framework Overview

Phases of the Value Impact Assessment

Each phase is described in more detail in the following sections.

VIA Phase 1: Value Opportunity Assessment

Figure 4. Value Impact Assessment phase 1 overview

Copyright FDF and other members of the EXPERIMEDIA consortium 2012

Change management issues

Copyright FDF and other members of the EXPERIMEDIA consortium 2012

VIA Phase 2: Value Opportunity Validation and Macro Modelling Phase

The outcome of phase 2 will be a model of:

VIA Phase 3: Industrialisation Assessment Phase

VIA Review Board Processes

Copyright FDF and other members of the EXPERIMEDIA consortium 2012

5. Methodological Approach towards Legal and Ethical Requirements

Privacy Impact Assessment (PIA) From a Privacy Impact Assessment on RFID

Copyright FDF and other members of the EXPERIMEDIA consortium 2012

to a general Privacy and Data Protection Assessment?

Copyright FDF and other members of the EXPERIMEDIA consortium 2012

PIA methodology development

Copyright FDF and other members of the EXPERIMEDIA consortium 2012

Copyright FDF and other members of the EXPERIMEDIA consortium 2012

Copyright FDF and other members of the EXPERIMEDIA consortium 2012

Figure 5. Privacy Impact Assessment methodology process

PIA Phase 1: Preparation

PIA Phase 2: Pre-assessment

Copyright FDF and other members of the EXPERIMEDIA consortium 2012

Answer Yes No Go to question 2 No PIA required

1. Is personal data being processed? 2. Is data linked to a natural person?

Full-scale PIA needed Small-scale PIA needed

Copyright FDF and other members of the EXPERIMEDIA consortium 2012

PIA Phase 3: Risk Assessment

Copyright FDF and other members of the EXPERIMEDIA consortium 2012

Figure 6. Sample data flow diagram38

Copyright FDF and other members of the EXPERIMEDIA consortium 2012

Figure 7. EU data protection privacy targets41

Copyright FDF and other members of the EXPERIMEDIA consortium 2012

PIA Phase 4: Documentation and Reporting

ICO, "Privacy Impact Assessment Handbook V2", ico.gov.uk, 2009, 40.

Copyright FDF and other members of the EXPERIMEDIA consortium 2012

PIA Phase 5: Review

Copyright FDF and other members of the EXPERIMEDIA consortium 2012

Copyright FDF and other members of the EXPERIMEDIA consortium 2012

Copyright FDF and other members of the EXPERIMEDIA consortium 2012

S-ar putea să vă placă și