January 2010

Using Action Research on the Process of Decision Support with VIP Analysis Software
a

Alecsandra VENTURAa,1 Luis C. DIAS b and Joo N. CLIMACO b PhD student in Management Science - Faculty of Economics, University of Coimbra b INESC Coimbra and Faculty of Economics, University of Coimbra

Abstract. This paper presents a summary of a research done in a Portuguese network of travel agencies, as part of a PhD Research in Multicriteria Decision Aiding - MCDA. This study is a first contribution to suggest an Implementation Model of Decision Support with the VIP Analysis software, including the proposal of using Cognitive Maps as a Problem Structuring Method (PSM) and using Multi-Attribute Utility Theory (MAUT) to elaborate additive value functions. Action Research was selected as a Research Method in this case, because it enables the assessment of acceptability VIP Analysis Software in Organizations, at the same time that it makes possible the implementation of a decision aiding process by the researcher. Keywords. MCDA, VIP Analysis Software, Cognitive Maps, MAUT, Action Research.

Introduction This study is justified primarily as a way to fill a gap in existing studies on the VIP Analysis Software [7], in particular about the lack of documented information on real situations of use of this software experienced by organizations that face decision problems of choice/selection. It also intends to provide to decision makers an implementation model that will guide the use of this Decision Support System (DSS) as new method which combines the following tools: The use of Cognitive Maps as a Problem Structuring Method; The construction of Value Functions using Multi-Attribute Utility Theory (MAUT); The field intervention using the Action Research Method, The use of VIP Analysis Software as an MCDA tool that can accept partial information on weights in Multicriteria Decision Problems. According to Montibeller [22], many authors have discussed the future of Multicriteria Decision Aiding (MCDA) Methods and indicated Action Research as an appropriate alternative to study MCDA (in this case, VIP Analysis) because it enables the
1

Corresponding Author: Tel.: +55 82 8893-4158 (Alecsandra Ventura), Email: alecsandra.ventura@gmail.com

January 2010

systematic investigation of a particular subject area while intervening in organizations. In other words, this approach helps the researcher to change the social system of organizations studied while investigating the impact of these changes, and generates knowledge based on them. For Belton & Stewart [3] only through Action Research an implementation of an MCDA Method can be genuinely investigated. Eden & Huxham [11] stressed the importance of documenting all the important information in interventions that use Action Research as a method of generating an active reflection on the results, bringing out new theories and providing relevant information for future interventions. These aspects of the Action Research Method will enrich the study and aim to make it an important source of information and reflection on all future interventions using the VIP Analysis Software as a tool for decision support. The activities of this research were developed within two cycles of activities that occurred simultaneously in an investigation by the Action Research Method, according to McKay & Marshall [20]: The Research Cycle (using Action Research Method) and the Problem-solving Cycle, by using Cognitive Maps and VIP Analysis, an MCDA software developed by Dias & Climaco that allows the evaluation of decision alternatives according to the Additive Aggregation Model advocated by Keeney and Raiffa [16], but not requiring the elicitation of precise values for criteria weights.

1. The Organization and the Decision Problem The Organization that participated in this study was Top Atlantico (TA), a private company from Portugal and currently holding the largest network of travel agencies in that country. The Organization was established in 2003 and currently has more than 80 travel agencies in Portugal (Mainland, Azores and Madeira Islands), Angola, Spain and Brazil. The company intended to expand its activities in Portugal by opening a new travel agency in the Center of the country or in Alentejo Zone. Therefore, it was necessary to select a location with great potential of demand for their products and services, to ensure the return on investment that will be done. The label of the decision problem studied was "Select a location for Top Atlantico new agency. This identification starts the Cycle of the Action Research with the activity of Document Structuring and Troubleshooting, according to McKay & Marshall [20].

2. The Field Research 2.1 Training the Decision-Maker (DM) on the subjects under study The Action Phase of the Research Cycle and the Problem-solving Cycle of the Action Research Method began in this study with the training of the DM on the subjects under study.

January 2010

According to Ortsman [24], one of the roles of the researcher in Action Research Method is to make available to the participants the knowledge and skills necessary to facilitate the discussion of problems, to record information collected during the application of this technique and, in collaboration with other participants, to design and implement actions related to the development of the project, generating ideas on the studied problem. Thus, the researcher provided training on the topics "Cognitive Maps", "Additive Model" and "VIP Analysis Software" to the Coordinator responsible for the Central Zone of Portugal. After training, the researcher and the Coordinator had structured the Cognitive Map using the Decision Explorer Software [4], had defined value functions and used VIP Analysis Software to process the information of the decision problem. The next topics present more details about these activities. 2.2 Modeling the Cognitive Map The researcher interacted with the DM to identify the Primary Assessment Elements (PAE) of the problem, which are the elements that the DM considered essential for the evaluation model being created. These elements were: population, number of families, evolution of the population in the last ten years, number of doctors per thousand of inhabitants, number of companies, number of companies of the secondary sector, number of companies of the tertiary sector, number of bank branches, value of the bank deposits, value of loans, amount of capital gains tax of individuals and Organizations, purchasing power of the population, and percentage of the purchasing power. These elements were converted into concepts and their respective psychological opposites were then defined, as advocated by Eden, Jones & Sims [13]. The researcher asked some questions to the DM to know the hierarchy between the concepts and the structure of the Cognitive Map. In this study, the Cognitive Map described causal relationships or influence, as suggested by Eden [10] and its construction followed the steps indicated by Montibeller Neto [23] and Bana e Costa [1]. In this process of analyzing the PAEs initially defined it was possible to identify the real concerns of the DM: The PAE "Population" - The DM wanted to evaluate the "Potential Sales" of the location, i.e., what really interested the DM was the proportion of the population that could buy the services of the Organization. The PAE "Number of Families" - The intention of the DM was to assess the trend of the population of the town (which could be examined by another index: the "Population Grown Rate). In the PAE Number of Doctors per thousand of inhabitants" the DM wanted to know about the number of people with higher education levels, which could be reflected by other indices related to the population's purchasing power.

January 2010

By the PAE "Number of Companies" the DM sought to evaluate the Potential for the Control of Seasonality, because if the TA has corporate clients in the town to be chosen, it would be possible to balance the company's revenue in times of low tourist demand. Thus, the researcher suggested the criterion Potential for the Control of Seasonality. As the research method selected is Action Research, in which the researcher has the freedom to actively participate in the study of the situation, she also suggested an assessment element that could be important in selecting the location and the DM had not thought about: the proportion of existing travel agencies in relation to the population of the town. The DM considered this was a good suggestion and the factor "proportion of travel agencies of the location in relation to population", which actually means "Competition, was then added. The researcher also suggested the analysis of the "Renewal Index of Population in Active Age" because it influences the index of purchasing power and the sales potential. Thus, at this stage some concepts originally defined by the DM were excluded and other concepts, not initially foreseen, were created.

Figure 1. Cognitive Map: Select a place to the new TA Agency In order to facilitate the analysis of the map, it was divided into clusters of similar concerns. Each cluster became a candidate Fundamental Point of View (FPV) and was tested to meet the properties for being a fundamental objective according to Keeney [16]. The final list of FPV is: 1. 2. Population: Number of residents in the location. Population Grown Rate (PGR): Population variation observed during a certain period of time, usually a calendar year.

January 2010

3.

4. 5.

6. 7. 8.

9.

Renewal Index of the Population in Active Age (RIPAA): Relationship between the population that is potentially coming vs who is leaving the labor market. Number of bank branches: Number of desks of banks of the town. Amount of bank deposits: Funds received by a monetary financial institution at the request of clients which are in the monetary responsibilities of these institutions. Amount of credit: Total value of financial assets made available when creditors lend funds to borrowers. Number of Companies: Number of companies in secondary and tertiary sectors existing in the locality. Purchasing Power (PP): Indicator Per Capita in the purchasing power index. It is a number that compares the purchasing power consistently expressed in different territorial units, in per capita terms and the average purchasing power of the country (which was assigned to the value 100). Competition: This indicator represents the number of travel agencies located in the town in relation to population.

2.3 Definition of Alternatives It was suggested to the DM to think about locations where Top Atlantico still has no agencies, trying to identify the places where he intuitively believed to exist greater potential for setting up a new TA travel agency (e.g places with a large population, places far from another Top Atlantico agencies, etc). Finally, the DM chose Beja, Braga, Castelo Branco, Covilha, Guarda, Santarem, Torres Novas and Viseu as potential alternatives to locate the new Top Atlantico Travel Agency. 2.4 Definition of Descriptors of Impact and Construction of the Value Functions The next step in our study was to to define Descriptors of Impact for each FPV and to construct Value Functions to obtain the criteria to the decision problem. A Descriptor (also called Attribute) is an ordinal scale to measure the Points of View and a Value Function is a tool used to help the DM to represent his preference according to an order of attractiveness [13]. To construct a Value Function, the first step was to determine the acceptance of the axioms of Multi-Attribute Utility Theory (MAUT) by the DM, and then to use elicitation techniques to ask questions about his preferences and to know the level of attractiveness between the performances of alternatives on each criterion. The constructed Value Functions allowed to convert the performance of all alternatives in each criterion into a valuation on the [0,1] interval, resorting to interpolation when necessary. Except the Competition criterion, all other criteria had increasing preference, with values to maximize. After conversion, in all value functions, the value 0 indicated the worst performance and the value 1 indicated the best performance.

January 2010

This conversion was based on the DMs evaluation of performance as reflected in the adjustment of pre-defined curves. To make this adjustment, the researcher told the DM that it is not necessary to have a linear conversion of performance on each criterion into value (An example of Value Function is presented on Figure 2). Some variation of performance could be considered more important than another and it is possible to consider satisfaction is achieved from a certain point onwards, which, once achieved would not have a major impact on the return or satisfaction for additional performance. These curves represent the satisfaction of the DM on the possible performance in each criterion.

Figure 2: Value Function of the criterion Renewal Index of Population in Active Age (RIPAA)

The overall value V(ai) of an alternative ai, according to MAUTs additive model, is obtained by weighing the values v1(ai),,vn(ai) achieved the by alternative at the multiple criteria by the respective value function weights (the scaling constants) k1,,kn: V(ai) = k1 v1(ai) + + kn vn(ai) 2.5 Definition of a ranking for the weights As required in this study, the criteria are compensatory in nature. The good performance on a given criterion could compensate for poor performance in another. As an example, a location with a smaller population, but with a greater purchasing power or a large number of companies might have to balance its overall performance, according to the order of importance. According to Marmol et al [17], the DM is often unable or unwilling to provide precise values for weights, but he may be able to establish certain relations that are interpretable in terms of marginal substitution rates between the criteria. So, the researcher explained the DM he could set weights for each criterion or just set the ranking of these weights, because the VIP Analysis Software would be able to deal with this type of information, or any other type of linear constraints on the weights. She also explained that the direct rating techniques should be avoided, because the value of these coefficients does not reflect the intuitive notion that the DM has about the importance of each criterion and that it would be more appropriate to use the Swings Technique or Equations of Indifference (Trade-offs) [18]. The DM preferred to use the Swings Technique and considered that the most important swing was the one

January 2010

for "Purchasing Power", because he considered that it would be the most relevant factor for the success of the new development in the selected location. However, the DM had some doubts to answer these questions about the other criteria and the researcher suggested him to revisit the Cognitive Map of the problem and to identify the criteria that have greater impact on results. The observation of actual performance presented in the value functions was also important to dispel the doubts and to choose the order of preference of the criteria. The ranking of the 0-1 swings yielded directly the ranking of the criteria weights: k8> k5> k2> k9> k3> k1> k7> k6> k4. 2. 6 The Use of the VIP Analysis Software Most studies on location problems using mathematical programming and are multiobjective models [14] [18]. Sometimes these models are inadequate to incorporate aspects relating to the subjective preferences of the DM and are difficult to measure. Such subjectivity requires the application of VIP Analysis Software or other methods [15] [21] which deal with different kinds of uncertain, imprecise and partially missing information, not only in location problems, but also in other types of choice problems using the additive model of MAUT. The additive model of MAUT to aggregate multi-criteria performances implies a compensatory mechanism between the criteria, where DMs are faced with the problem of identifying a replacement rate (trade-off) of one criterion over another. These tradeoffs are reflected in the ratios between criteria weights: e.g., if k2 is twice k1, then one unit in value function v1(.) can be traded-off for 2 units of v2(.). VIP Analysis [7] is multicriteria decision support software to evaluate a discrete set of alternatives in choice problems. Based on the additive model for the aggregation of value functions, its main characteristic is that it does not require the DM to indicate precise values for the trade-offs between different criteria. Rather, it can accept imprecise / partial information on the criteria weights k = (k1, k2, ... kn) (namely rankings, intervals and other linear constraints). VIP Analysis may be used to discover robust conclusions, i.e. those that hold for every accepted combination of the weights, and to identify which results are more affected by the imprecision. The researcher started by entering all data of this decision problem (criteria, alternatives and the respective values of these alternatives with performance on a scale of 0 to 1, as well as the ranking constraints on the weights) into the software (Figure 3). Based on this information, VIP Analysis presented the results summarized in Figure 4.

Figure 3. VIP Analysis Software Data

January 2010

Figure 4. VIP Analysis Software Summary The DM did not have to give a weight for each criterion: he had just determined the order of importance and had entered this information in "Constraints". With this information (let K denote the set of weight vectors that comply with the constraints), the software uses optimization (a linear programming algorithm) to compute the maximum value and minimum value that each alternative ai may achieve given these constraints: min{V(ai): (k1,,kn) K} and max{V(ai): (k1,,kn) K}. In the "Summary" page, these results appear in the column "Min Value (Minimum Value) and Max Value (Maximum Value). The software also makes pairwise comparisons between the alternatives to compute the maximum difference of value between them. The "Confrontation" table in Figure 54 indicates, in each cell, the maximum difference max{V(arow)-V(acolumn): (k1,,kn) K}. For instance, it can be read that Beja can win over Castelo Branco by a difference of 0.3 (for some vector of weights), but Castelo Branco can win over Beja by a difference of 0.275 (for another combination of weights). In the column referring to Guarda the negative values indicate that Covilha and T. Novas can never win over Guarda, and the zero value indicates that Santarem can at most equal Guarda. Therefore, Covilha, T. Novas and Santarem are dominated by Guarda (Covilha being dominated by other alternatives also).

Figure 5. VIP Analysis Software Confrontation

January 2010

The line "Max Regret" (maximum regret) presents the maximum difference of value by which an alternative can lose to another. The lower the Max Regret is, the better is the alternative. A Max Regret equal to zero or even negative would mean that the alternative was better than all other for all vectors of weights complying with the given ranking. The Range Map (Figure 6) depicts the lowest value of each alternative (Min Value) and the highest value of each alternative (Max Value). The difference between the highest and lowest value shows how much each alternatives value depends on eliciting further constraints on the weights: uncertainty for Guarda, for instance, is much less than uncertainty for Beja. 2.7 Analysis of Results and Recommendations The VIP Analysis Software shows that alternative Covilha, which has the worst performance in the most important criterion (Purchase Power) and the largest Max Regret, is Absolutely Dominated. This means that its maximum value is below the minimum value of other alternatives, which would recommend exclusion of this alternative from the analysis. At this moment, the DM showed some disappointment, because of his "sympathy" for this town, which even has good customers. The researcher proposed to simulate some changes in the ranking of the weights to study the behavior of alternative Covilha, but the DM stressed that as the Purchasing Power is the main criterion and Covilha has the worst performance in that criterion, it was not worth doing this simulation.

Figure 6: VIP Analysis Software Range Map

January 2010

The alternatives Portalegre and Santarem were also indicated as dominated by Guarda. Santarem, however, has a Max Regret very low, equal to the best alternatives (Guarda and Castelo Branco). In this situation the DM asked what could be remade in the analysis so that the alternative Santarem might get better results without changing the ranking of the criteria. Santarem has a good performance on most criteria, and it would stop being a dominated alternative if a tolerance of -0.01 was considered. The "rescue" of this alternative among the dominated alternatives appealed the DM, because he was intuitively aware that this location would be a strong candidate to be one of the Top Atlntico Agencies. Thus, the software showed the following alternatives with better performance, in order of minimum value are: 1st: Guarda, 2nd: SantaremS, 3rd: Castelo Branco. The researcher recommended the selection of either of these alternatives in order to attend the main objectives of Top Atlantico in their marketing strategies because these are places with great sales potential and great potential for control of seasonality (as typical of area of Tourism). The researcher explained the DM that Guarda presented the smallest amplitude between its minimum and maximum values (see Figure 6) and that small amplitude corresponds to greater robustness in the sense that the results obtained by the alternative Guarda would have less of an impact if there was variation in weights that reflected the preferences of DM. She also called the DMs attention about Beja because it might also be an interesting alternative, as it can achieve the maximum value (it has value 1 on the 8th value function, with k8 ranked the highest weight). However, Beja was discarded because it has value 0 on the fifth value function, with k5 ranked the second highest weight and because it has a small robustness. 3. Evaluation of the effects of the intervention The Evaluation of the Effects of Actions and the Evaluation of the Effects of Intervention were the final stages of the Problem-solving Cycle and the Research Cycle in the Action Research Method. The researcher made these evaluations by using an application form in which the DM could evaluate the process of decision support and the use of VIP Analysis. According to the DMs evaluations, this study brought a new and interesting knowledge about a structured decision making process. However, the Cognitive Maps Method and the use of VIP Analysis were considered a complex process by the DM. He deemed that he only would use these tools again if he had again an assistance of a specialist to guide them into a new process of decision support. 4. The Main Contributions of this study

January 2010

Prior to this study Top Atlantico used to select locations for its new agencies by analyzing the factors mentioned in Section 2.2, considered as criteria. However, no method was used to aggregate the performance of alternatives in each criterion in order to obtain a comprehensive and consolidated value of each alternative. In this intervention it was possible to identify different statistical indices than those originally imagined by the DM, through the technique of Cognitive Maps which ensured a more effective analysis of the problem. New concepts were created and other concepts were excluded as a result of the dialogue between the researcher and the DM in setting the hierarchies of concepts. As an example, we can mention the PEA "Number of Families." By asking the DM why was it important to know the number of families in the locality, the researcher understood that that the DMs real concern was to know whether there was a greater likelihood of people taking up residence on site (and therefore not risk a population decline that would affect the demand for products and services in the location). Therefore, it was possible to identify a statistical indicator more effective for the analysis of population, which was the "Population Grown Rate. It was also important to provide the DMs a comparative overview of the alternatives, allowing an intensive analysis of the performances of all of them. This was possible through the aggregation of performance values of alternatives and their processing and analysis using the VIP Analysis Software. The DM had an opportunity to use a structured and effective methodology to support decision-making. This research was the first intervention with the intention to develop an implementation model to VIP Analysis Software and it gave the researcher the opportunity to combine theoretical knowledge and practical experience that comes from use of these tools in this study, consolidating and improving her working methods, to be applied in subsequent studies. Other two interventions are underway at this moment and the consolidation of these experiments, with better methods will provide the basis for the composition of the VIP Analysis implementation model, which can then be generalized to future interventions. References
[1] Bana e Costa, C., Structruration, construcion et explotation d'une modle d'aide la dcision, PhD Thesis of Universidade Tcnica de Lisboa. . Lisboa, PT, 1992. [2] Belton, V., The Need for Interaction and Integration. Journal of Multicriteria Decision Analisys 6 Editorial (2001) , 127-128. [3] Belton, V., & Stewart, T. Multiple Criteria Decision Analysis, Kluwer Academic Publishers, UK, 2002. [4] Brightman, J. R., An introduction to Decision Explorer, Banxia Software Ltd, (2002) (available online at http://www.banxia.com/dexplore/pdf/DEIntro1.pdf) [5] Climaco, J., & Dias, L., An Approach to Support Negotiation Process with Imprecise Information Multicriteria Additive Models. Group Decision and Negotiation , 15 (2006), 171-184

January 2010

[6] Dias, L.; Climaco, J., Dealing with imprecise information in group multicriteria decisions: a methodology and a GDSS architecture. European Journal of Operational Research , 160 (2005), 291-307. [7] Dias, L., & Climaco, J., Additive Aggregation with Variable Interdependent Parameters: The VIP Analysis Software. Journal of Operational Research Society, 51 (2000), 1070-1082 [8] Dias, L., A note on the role of robustness analysis in decision-aiding process, Instituto de Engenharia de Sistemas e Computadores de Coimbra INESC, Coimbra-PT, 2006. [9] Easley,R.; Valacich,J.; Venkataramanan,M., Capturing group preferences in a multicriteria decision., European Journal of Operational Research, 125 (2000), 73-83. [10] Eden, C., Cognitive Mapping. European Journal of Operational Research, 36 (1988), 1-13. [11] Eden, C., & Huxham, C., Action Research for Management Research, Journal of Management , 7 (1996) , 75-86. [12] Eden, C., Jones, S., & Sims, D., Messing about problems., Pergamon.Oxford, UK, 1983. [13] Ensslin, L., Montibeller Neto, G., Noronha, S., Apoio Deciso Metodologias para estruturao de problemas e avaliao multicritrio de alternativas, Editora Insular, Florianpolis/Santa Catarina Brasil, 2001. [14] Erkut, E., Neuman, S., A Multiobjective Model for Locating Undesirable Facilities, Annals of Operation Research, 40 (1992), 209-227. [15] Gamboa, G., Munda, G., The problem of windfarm location: A social multi-criteria evaluation framework, Energy Policy, 35 (2007), 1564-1583. [16] Keeney, R. & Raiffa, H., Decisions with Multiple Objectives: Preferences and Value Tradeoffs, Wiley, New York - USA,1976. [17] Keeney, R.L., Value-Focused Thinking: A Path to Creative Decisionmakin, Harvard University Press, Cambridge, UK:, 1992. [18] Malczewski, J., W, Ogryczak., An interactive approach to the centre facilyt location problem: locating pediatric hospitals in Warsaw, Geographical Analysis, 22 (1990), 244-258. [19] Marmol, A., Puerto, J., Fernandez, F., The Use of Partial Information on Weights in Multicriteria Decision Problems, Journal of Multicriteria Decision Analysis, 7 (1998), 322-329. [20] Mckay, J., & Marshall, P. , The Dual Imperatives of Action Research,. Information Technology & People, 14 (2001), 46-59. [21] Menou, A., Benallou, A., Lahdelm, R., Salminen, P., Decision support for centralizing cargo at a Moroccan airport hub using stochastic multicriteria acceptability analysis, European Journal of Operational Research, 204 (2010), 621629. [22] Montibeller, G., Action Researching MCDA Interventions. 49th British Operational Research Conference (OR 49), Edinburg: OR Society. (2007) [23] Montibeller Neto, G., Mapas Cognitivos: Uma Ferramenta de Apoio Estruturao de Problemas, Dissertao de Mestrado da Universidade Federal de Santa Catarina, Santa Catarina-Brasil, 1996. [24] Ortsman, O. , Changer le travail, les expriences, les mthodes, les conditions de l'exprimentation sociale, Paris: Dunod., 1978. [25] von Winterfeldt, D. & Edwards, W., Decision Analysis and Behavioral Research, Cambridge University Press, New York USA, 1986.