The Architectural and Group Development Method: an Experimentation

Moiss Gonzlez-Garca1,2
2

Ana Mara Martnez Enrquez1

1

CENIDET Centro Nacional de Investigacin y Desarrollo Tecnolgico Int. Internado Palmira S/N Cuernavaca Morelos, 62490, Mxico (52) 777-312-2314

CINVESTAV-IPN - Centro de Investigacin y Estudios Avanzados del I.P.N. Av. IPN No. 2508 Mxico D.F., 07360, Mxico (52) 555-061-3800

moises@cenidet.edu.mx, mgonzalez@computacion.cs.cinvestav.mx ABSTRACT

Software development processes recognize importance of software architecture, but it seems not to have influence on software design and implementation. Current software industry requires an agile development process performed within organized groups, but it could better promote and integrate groupwork. We propose the AGD (Architectural and Group Development) software development method allocating a product set with explicit software architecture as primary element, and integrating in a process four group-work modes (Team, Collaborative, Cooperative, and Control) as secondary element. On the contrary with RUP, AGD is a method that gives more importance to products, and that subordinates the process to the results being produced. Complexity of the process is reduced eliminating the phases, and obtaining any required result with the same systemic process. We assume better quality products can be obtained from the result of: 1) the importance given to products and their architecture, and 2) the synergy obtained integrating group-work modes in a systematic process. This paper presents the design and the results of an experimentation performed to quantitatively evaluate the AGD method. Its behavior is shown relatively to the RUP de facto software development environment. The presented results confirm our hypothesis by which a team working with the AGD method will release software products with less defects than a team working with RUP.

ammartin@cinvestav.mx

Categories and Subject Descriptors

D.2.9 [Software Engineering]: Management life cycle, programming teams, software process models (K.6.3 [Management of Computing and Information Systems]: Software Management software development, software process).

General Terms
Management, Measurement, Documentation, Experimentation.

Keywords
Software development method, agile method, results driven process, collaborative work, cooperative work.

1. INTRODUCTION
In this paper we focus on how are used quantitative techniques in an experiment to validate evolution of the current de facto standard for software development: the UP (Unified Process) model [19] and its commercial version the RUP (Rational Unified Process) software development system [26] [27] [21]. Any evolution in software development is important because by now, more complex software is needed for every sector of economy to support group-work, to include more areas, and more functionality. Because of these characteristics, software development oversteps the bounds of individual skills. This status gets worse because software importance, for organizations, has been moving to the center of attention. The goal of this research consists in setting up a software development method centered on a product with explicit software architecture1, with a product life cycle2 oriented to incremental3

Software architecture is a set of rules that define the software structure and the relationships among its parts [17]. Product Life Cycle is the period of time that begins when a software product is conceived and ends when the software is no longer available for use [30]. Incremental development is a technique in which definition of requirements, design, implementation and testing phases occur in an overlapping and iterative manner, resulting in incremental completion of the software product [16].

completion of results, and that integrates group-work in an incremental, cooperative, adaptive, and direct process. We achieve this goal proposing an alternative software development environment: the Architectural and Group Development (AGD) method4. In order to highlight the improvements and to establish their advantages, we present the AGD design, the performed experiment, and the results. The result analysis considers components of RUP and AGD, highlighting relations between them. In this paper, we try to put in evidence AGD solves some of the important inherent problems of contemporary software development [11] [12] [1]. In section 2, we present the way by which AGD synthesizes a different structure eliciting a deep change to RUP. It proposes an alternative method, allocating work products5 as primary elements, and integrating a process with four group-work modes. Section 3 describes the experiment setup, procedure and measures done to follow AGD and RUP. The results shown in sections 4 and 5 present how these two models validate the experiment hypothesis. These results show that AGD allows practitioners designing and producing good quality software products by means of a suited and efficient process. Finally, in section 6, we present our conclusions and the possible extension of this study.

RESOURCES-D defines how the elements are distributed across the network; and the LOGIC-D describes the software product architecture and the organization. AGD establishes how a software development team can apply empirical approaches for software development. Its principal practices are: to interactively develop software, to manage requirements, to use a model driven architectural style, to build a reference model, to specify and to verify the software quality, to control changes applied on software, and to take benefits from the synergy of four group-work modes. The AGD method allows developers engineering of a product, following an orderly process. Describing AGD along two axes (see Fig. 1), the horizontal one shows the Product Set, synthesized with explicit software architecture. The vertical axis represents the Model Driven Process (MDP) that is a systematic and iterated group-work process. RUP is a generic two-dimension process described along two axes too. However, the horizontal axis shows the dynamic aspects of the process (e.g. phases, iterations, and milestones), and the vertical axis represents the static aspects of the process (activities, artifacts (UML models), workers, and disciplines). The ProductSet is an orderly set, in which the element sequence (ProductS1Set) suggests the natural order to produce them. As shown in Fig. 1, it contains three models of MDA focusing 1) on Computation Independent Models (CIM), 2) aiming to Platform Independent Models (PIM), and 3) providing Platform Specific Models (PSM). More, the Product Set includes two complementary sub-sets: the Implementation Model (IM), and the Operation Model (OM). Further, the PS is a set in which all elements are known and are possibly developed in different sequences. In case of missing elements, users are notified and then have to consider them when defining strategy and planning activities. The Product Set follows a four level hierarchy: ProductSet is the root of an inverted tree organizing the four levels: ProductS1Set (see Fig. 1), ProductS2Set (see Fig. 3), ProductS3Set, and Work Products. The ProductSet has to be completely engineered7, deciding when and how the completion status is reached. This status has to be established by agreement between the client and the project team. All ProductSet models are produced using the Model Driven Process (MDP) that is a generalized iteration process of four group-work modes: Common Team, Collaborative, Cooperative, and Control. In contrast RUP is a two dimensions process: the horizontal dimension includes a sequence of phases (Inception, Elaboration, Construction, and Transition), and the vertical one considers a sequence of Core Disciplines which names have similar meanings (Business Modeling, Requirements, Analysis & Design, Implementation, Test, and Deployment). Additionally, RUP defines other set of Supporting Disciplines (Configuration & Change Management, Project Management, and Environment) to complement the process.

2. THE AGD METHOD

The AGD method (Architectural and Group Development) is built up considering two dimensions (See Fig. 1): 1. A Product Set (PS) [9] is a hierarchy of sub-sets whose elements are 1) models made to develop the targeted software, and 2) measured in units of size (e.g. lines of code, lines of text, paragraphs), A Model Driven Process (MDP) [10] that is composed of four group-work modes: a common team-work, and three complementary group-work modes, measured in units of effort (person hours).

2.

AGD assigns and manages software productions (ProductSet or PS) and responsibilities in a development organization. The goal of the AGD Model Driven Process6 (MDP) is to produce (within a predictable schedule and budget) high-quality software that satisfies the needs of end users. This process is software architecture centered, since it is driven by the ProductSet synthesized with models that are adjusted to a reference model of information systems, with an explicit architectural style. The software product is an element contained in AGD Product Set, which follows two models representing different views of the software architecture: the MDA style [23], and the CWSLR model (Cooperative Work Storage, Logic, and Resources) [8] that depicts a software product or system. CWLSR is also part of our research investigations, and defines a three-component architecture style, built by: the STORAGE-D containing the product elements stored in a stable secondary storage; the
4

Method is an orderly process used in the engineering of a product [16]. Work Product is an artifact used, produced or changed by a process [18]. Process is a set of interrelated activities that transforms inputs into outputs [15].
7

Engineering is the application of a systematic, disciplined, quantifiable approach to structures, machines, products, systems or processes [16].

PS MDP
Common Team (SubProcesses) I T SPE Requirements SP-Analysis & R Design SP-ImplementaA tion T SP-Test I O N Colaborative (development) Cooperative (revision) Control (status) SP-Postmortem SP-Launch & Strategy SP-Planning ProductS1Set CIM ProductS1Set PIM

ProductSet
ProductS1Set PSM ProductS1Set IM ProductS1Set OM

Figure 1 - The Architectural and Group Development (AGD) method

Instead of defining additional process types as provided by: 1) RUP with its Supporting Disciplines [26], 2) the Capability Maturity-Model, with four process categories and 22 process types [25], 3) the ISO standards 15504 and 12207, with five process categories and 25 process types [18] [15] additional work products or models of the Product Set are included for complementary purposes (e.g. configuration management or project management). Thus, in order to control quality of product and process, 23 data-capture forms are considered and added to the AGD method. These additional products are always elaborated using the same systematic group-work modes as used

by MDP. In MDP, the Common Team mode gives context to the other three group-modes; it includes seven asynchronous sub-processes that are performed by collaborators playing different roles. The first and the last sub-process are fixed, but the other five do not require a well-established sequence. This MDP characteristic allows the global process being flexible, incremental, modifiable and adaptable. The seven sub-processes are: SP-Launch&Strategy, SP-Planning, SP-Requirements, SP-Analysis&Design, SPImplementation, SP-Test, and SP-Postmortem (see Fig. 1).

PROJECT

MODEL DRIVEN PROCESS (MDP)

MANAGEMENT (CONTROL) 1

CONTROL GW (CONTROL)

DEVELOPMENT (COLLABORATIVE) 2

DEVELOPMENT GW (COLLABORATIVE)

TECHNICAL REVIEW

TECHNICAL REVIEW GW (COOPERATIVE)

PROJECT MANAGEMENT COMMON TEAM GW (SUB-PROCESSES)

Figure 2 - The Model Driven Process (MDP)

The sub-process SP-Launch&Strategy establishes what models other sub-processes produce, the sub-process ordering, and the scope of their associated models. In order to set the strategy, AGD suggests to consider the following system structure: 1) System isan assembly of Products, 2) Products include Components, 3) Components are compositions of Modules, and 4) Modules use objects. Each iteration terminates releasing an internal or external product (executable, if possible), which grows incrementally from iteration to iteration until the final system is achieved. The last sub-process, SP-Postmortem purpose is 1) to gather, analyze, and record project data, 2) to evaluate the team members and their associated role performance, 3) to identify ways to improve the next cycle process, and 4) to produce the cycles report. For each sub-process, a model process describes who and how to do what. The quantity and scope of models, produced in the subprocesses, differ for each iteration, and depend on the ProductS1Set produced. This characteristic is shown in Fig. 1 with colored triangle areas. The left acute angle represents the process initialization and the opposite right side shows the quantity of the final results. The triangle with the largest opposite right side represents the sub-process that produced more models with larger scopes. Group-work, used during the MDP process, constitutes the natural way to model the targeted project specifying four modes that can be executed concurrently as shown in figure 2: The collaborative mode [31] [32] [22] - used to

collaboratively develop the models or products, The cooperative mode [3] [4] [14] - used to review and evaluate the models or products, The control mode [29] - used to report and to plan the development/management efforts to produce any model or product, The Common Team mode [13] used to perform as a team. As sub-processes, it uses a modified version of the set of Process Steps (see Fig. 1) from the Team Software Process, and gives context to the three other modes.

MDP is dynamically driven by events as shown in Fig. 2. As example, a cooperative review is triggered when any one of the two following events occur: a) a work product is terminated (made in Collaborative mode), or b) the termination of a cycle (execution of the complete set of MDP team-work sub-processes). The Control group-work mode, performed during a meeting to review the project status, may be scheduled to a cooperatively established date and/or at a periodic time. The participant defines the behavior and the responsibilities of an individual or a group of individuals producing as a team. All participants always have the role of Team Member (using the Common Team mode). Moreover, a participant can act several

Experiment: Requirements Folder

AGD: Vision/Planning Folder (VF) CIM-(CWSP Initialization) CIM-(Planning/Vision) CIM-(Scope/Priority) CIM-(Results from revision of VF) AGD: Requirements Folder (RF) CIM-(Existing business) CIM-(Component identification) CIM-(System prototype integration) CIM-(System requirements) PIM-(Reverse engineering) PIM-(Forward engineering) PIM(Software prototype integration) PSM-(Requirements restrictions) PSM-(Acceptance test specification) PSM-(Software requirements) PSM-(Results from revision of RF)

Experiment: Design Folder

ADG: Design Folder (DF) PSM-(Component architecture) PSM-(Integration test specification) PSM-(Architecture analysis) PSM-(Work breakdown structure, and unit Assignment) PSM-(Results from revision of DF)

Experiment: Users Folder

ADG: Operation Folder (OF) IM-( Initial operation) IM-(Results of integration test) OM-(Resources identification) OM-(Distribution) OM-(Product release) OM-(Results from acceptance test) OM-(Assessment results)

Experiment: Implementation Folder

AGD: Unit Development Folder (UDF) Unit selected software requirements (from RF) Unit selected general design (from DF) IM-(Detailed design) IM-(Unit test specification) IM-(Results of unit detailed design revision) IM-(Functions list) IM-(Source code) IM-(Results of unit test) Defects report IM-(Results from revision of UDF)

Experiment: Metrics (forms)

AGD: Forms Time Reporting Log (CW-LOGT) Defect recording Log (CW-LOGD) Strategy Form (CW-STRAT) Product Delivery and Feedback (CW-PDF) Weekly/Cycle Status Report (CW-WEEK) Individuals tasks schedule Configuration Change Request (CW-CCR) Configuration Status Report (CW-CSR) Inspection Report-Detail (CW-INS-DET) Inspection Report_Sum (CW-INS-SUM)

Figure 3 - Correspondence between Experiment folders and AGD folders.

roles at the same time, as a minimum one roll for each groupwork mode. Those roles define how participants carry out their work within activities. The participants have to produce results using process management along with the social competencies that match with the required competencies of given roles. This differs from RUP that considers technical competencies when defining roles. Each group-work mode defines and uses a specific set of roles:
Common Team mode: Team Member, Team Leader, Planning Manager, Support Manager, Development Manager, Quality/Process Manager, Client (external), and Project Manager (external) Collaborative mode: Driver, Co-driver, and Assistant Cooperative mode: Facilitator, Writer, Reviewer, Producer, and Observer Control mode: Leader, Recorder, and Participant

required for developing her/his product (e.g. build PIM-(Software Prototype Integration) before development of PIM-(Forward Engineering)). This example of fast prototyping is quite frequent because there are several fourth generation languages available. Furthermore users can state a strategy to develop the complete Product Set in several cycles. In order to give an intuitive interface to developers, all produced ProductSet models can be organized, for printed output, in five documentation folders (see Fig. 3).

3. EXPERIMENTATION USING RUP AND AGD

To validate the evolution of the current software development environment, we carried out a controlled experiment. As a guide for our research, we established the following main hypothesis:
Hypothesis: High quality software products can be obtained by means of a group-work process oriented to build work products following the MDA and CWSLR models. The team synergy provided by group-work modes achieves better performance.

For its Common Team mode, AGD uses a customized version of roles from the TSP [13]. Participants, doing roles, perform activities that import and export work products (artifacts). A work product is a piece of information that is produced, modified, or used by a process. We started our design of the data-capture forms, a) modifying the 21 forms from TSP, b) replacing the concept of phase with the one of ProductSet models, and c) finally including the five folders output structure of AGD (see Fig. 3). Main data-capture forms, (see table 1), are mentioned in the next section while we describe the experimentation.

In order to validate this hypothesis, we developed and tested a quite realistic experimentation: 12 graduate students in computer science were randomly separated in four three-members teams. Two teams used the AGD method while the other two teams worked with RUP. The experiment goal consists in measuring and analyzing both the systemic software process and the produced software. The metrics used were identified using the Goal Question Metric (GQM) technique [6].

The figure 3 presents the ProductSet showing only the two levels 3.1 Setting Up the Experiment ProductS1Sets (e.g. CIM) and ProductS2Sets (e.g. Planning). This The studied differences are focused on the expected behavior of organization is flexible, and users are free to develop all the the software development process based on the environment ProductS2Sets within a sub-process following any order. Moreover, they can plan a special sequence that seems to be Table 1 Data-capture forms used during the experiment
Associated Event Continuously used Experiment Form Time Recording Log Defect Recording Log Cycle Beg.-End Week Beg.-End Strategy Product Feedback Delivery Purpose To record the time spent on each project task To hold data on the defects found and corrected To record strategic decisions AGD Form Time Recording Log (CW-LOGT) Defect Recording Log (CW-LOGD) Strategy Form

(CW-STRAT) and To record feedback response time and comments Product Delivery and Feedback (CWfrom client PDF) To prepare the weekly status report To estimate the development time, and product size for individuals tasks To submit an item to the configuration control board Configuration Change Request (CCB) for inclusion in the product baseline (CW-CCR) To provide weekly status information on the Configuration Status Report software configuration management (SCM) system (CW-CSR) To gather inspection / revision data in detail Inspection DET) Report-Detail (CW-INSWeekly/Cycle Status Report (CW-WEEK)

Weekly/Cycle Summary Individuals tasks schedule Change Request Configuration Status Report Inspection Report-Detail Inspection Report-Summary

Finish development of Product / Cycle

To analyze and to summarize inspection / revision Inspection Report-Summary data (CW-INS-SUM)

assigned to each team. Basically, we can observe distinctions exist among the groups from the process and its products points of view. The experiment was conducted within an academic environment where it was possible to simulate key elements (e.g. interaction with the client, and acceptance of product) of a production environment. There are four replications of the basic task; a separate software development team concurrently and independently performs each of them. The teams are named AT and RT, and respectively use AGD and RUP. Experiment is conducted in the field of two equivalent courses of Master level. All students attend a two hours session that includes a lecture on software experimentation, and on the experimentation design. The second two hours session includes a lecture on the methodology each team will apply. The participants answered a questionnaire focused on their previous software development experience. There is a Web page where teams can access the lecture contents and the associated material. More, every group can use the Rational Rose Enterprise Edition 2003 software in order to develop their models. Additionally, a professor is present to answer any question raised by AT or RT members. After the experiment, participants have to answer to a final questionnaire centered on the usability of the software development environment. Moreover both groups perform a dynamic activity to carry out the experiment process, and to evaluate each method (benefits and disadvantages). The data-capture forms (see table 1) can be both expressed using AGD and RUP, but RUP does not include project data-capture forms. The IBM Rational Suite can include external software and plug-ins to produce some status reports required to manage a software project [33] [34]. A partial list follows: ProjectConsole, RequisitePro, ClearQuest, ClearCase, SUMMIT Ascendant, and Microsoft Project. The first column of Table 1 shows events which occurrence triggers the filling and collecting of data from each capture form. The second column shows the names of capture forms used in the experiment by the participants. The third column describes the uses of the capture forms. The fourth column shows the names of the capture forms used by teams following AGD that approximately collect the data required. In order to validate and to compare RUP and AGD, a predefined set of five folders is required. Fig. 3 presents the predefined AGD folders that organize the documentation of the resulting software system and the corresponding experiment folders. The generic RUP process does not define how documentation can be organized. Weekly and when cycles are finished, participants of each group give to researchers a version of the required folders.

A library information system must provide five on-line operations (for users and staff), to control the book catalog and the borrowing operations, The system must offer functions to rapidly perform catalog searching and updating: it will avoid long-time check-out, it will make on-line book search a viable alternative to card catalogs, and it will easily handle updates and corrections by staff users to a potentially large collection. The system must also enforce four integrity constraints concerning the collection and the borrowers. The system development: 1) considers design variations, and 2) includes the application interaction with database, the centralization of system, and the type of database.

Using Albrechts function points [2] [24] [5], we have to take into account and quantify the targeted software product size. These points make possible the quantitative evaluation of how the products obtained in the experiment satisfy the required functionality. Albrechts formula to calculate the Unadjusted Function Point count is expressed as: UFP = IN*WI + OUT*WO + INQ*WQ + FILE*WF + EINT*WE From the analysis of the problem statement, we identified IN = 5 external inputs, OUT = 9 external outputs, INQ = 16 external inquires, FILE = 5 internal logical files, and EINT = 2 external interface files. Applying Albrechts formula with such values and the following average complexity weights: inputs WI=4, outputs WO=5, inquires WQ=4, internal files WF=10, and external interfaces WE=7, gives: UFP = 4*4 + 8*5 + 15*4 + 4*10 + 2*7 = 170 FP This function points have to be adjusted for the technology platform required by the client: a Web application. For this kind of application, we give the following values to the 14 characteristics (C1 C14) included in Albrechts Adjusting Factor (AF): data communication = 5; distributed processing = 5; performance critical = 5; online data entry = 5; file updated online = 5; facilitate change and easy of use = 5, and remaining eight characteristics with value 3. This gives: AF = 0.65 + 0.01 *

C = 0.65 + 0.54 = 1.19

i i =1

14

This value for AF to compute the Adjusted Function Points count (AFP) from the problem statement, gives: AFP = UFP*AF = 170*1.19 = 202.3 FP AFP is used, in the interpretation of the experiment (see section 5), comparing it with function points evaluated for the final software products, and with function points omitted by development teams.

3.2 Development Assignment

Every team task consists in designing and developing a software system giving a solution to the library problem [28]. This problem statement is summarized, without detailed options and constraints as follows:

3.3 Procedure of Experimentation

The teams have 28 working days to complete their software developments. The performed activities are:

Step 1 Using AGD or RUP, team members apply their assigned method/process, and day after day they handle the data-capture forms, and work products. Step 2 They terminate and install the software product on the clients personal computer. Step 3 They provide the documentation folders to researchers. Step 4 They perform the validation/acceptance tests of the delivered system, using the client personal computer.

section 3.2) provides a quantified value of 202.3 FP of the required functionality. The function points are used to evaluate the accomplished software products: these values are named M7_T, and can be compared with the M3_A omissions identified by client. The adjustment factor (AFP) was 1.19 for AT(1), since it developed a Web application, and was 1.04 for other three teams because they did not consider a Web environment. The total actual effort, named M5_T, is reported as the amount of hours spent by team members on development, and is collected from capture forms Time Recording Log, and Weekly/Cycle Status Report. These values are required to compute the productivity rates, and they define a continuous process improvement. The defects detected during the acceptance test, named M1-A, are obtained from capture forms: Defect Recording Log, and Product Delivery & Feedback. This metric is generated after the software product is released to the client, and is important to evaluate the product and process quality, as perceived by client. In table 2, an asterisk (*) is associated with the best value of the metric of each row. However, we have to consider the other metric values to produce a correct evaluation of the product or the process, and then identify improvement opportunities. For example, the best value of the M2_CN metric (New or Modified Code) is 743 LOCs, developed by AT(1). However, to have a correct global perception, you have to relate this value with others, for example the value of 94.0 FPs for M7_T product functionality, the 6 M1_A defects found in acceptance test, and the 167:59 M5_T total real effort in person hours. In our example, AT(1) delivered the software product with less LOCs and less defects, but they implemented 94 function points. If we consider the team RT(2) and then compare its metrics with those of AT(1), we can see the M2_CN (New or Modified Code) value is 1305; it is approximately the double of the AT(1) score, with approximately a double value for M7_T = 205.92, and a double value for M1_A defects = 13. Considering this set of metrics (M2_CN, M7_T, and M1_A), the teams AT(1) and RT(2) can be considered as equivalent from the performances point of view.

3.4 Evaluation Metrics for the Process and the Software Product
In order to quantitatively evaluate the quality of the software product, and its development process, we use 6 metrics: M1_A - Detected defects in the product (in acceptance test). M2_TC - Size of product (Lines Of Code (LOC) of total product). M2_CN - Size of product (LOCs of new or modified code). M3_A - Omissions identified (in acceptance test). M5_T - Actual effort (Total effort). M7_T - Size of product (Total function points in product). Table 2 presents the values of the above metrics captured during the experiment. The second row of table 2 shows values of the experience in software development. These values are computed from a questionnaire and goes from 1.7 to 3.0 (in a range from 1 to 5), meaning that the lowest experience was gotten at class, and the higher experience was gotten in a class project. There is practically no team that has industry experience in software development. The total size of product, called M2_TC, is reported in Lines Of Code (LOC), that accumulates reused or automatically generated lines of code with new and modified lines of code. M2_CN represents the new or modified LOC. The amount of reused or automatically generated code (RC) is calculated subtracting M2_CN values to M2_TC. In this study, the ratio of automatically generated code to new or modified code (RC / M2_CN), is in the range that goes from 1.10 for AT(2) to 1.64 for AT(1). The Product Functionality estimates the product size measured in function points. This measure used at the problem statement (see
AT(1) Web-JSP + Java + MySQL, MySQL -front, JDBC 2.3 1965 * 743 * 94.00 167:59 120.19 6*

3.5 Statistical Approach

The development methodology is an experimental treatment

Table 2 Collected metric values

Metrics Development platform Previous experience in software development Size of product M2_TC - Total Code (LOC) M2_CN - New or modified Code (LOC) M7_T - Product functionality AFP (FP, Function Points) M5_T - Total Actual effort (person hour) M3_A - Omissions identified by client: In acceptance test FP) M1_A - Defects found in acceptance test AT(2) Visual Basic + Access 3.0 5539 2635 174.72 211:27 119.00 10 RT(1) Visual Basic + Access + Crystal Reports 1.7 4729 1820 145.60 105:52 * 92.82 13 RT(2) Visual Basic + MySQL, MySQL-front, DriverODBC of MySQL 2.1 3356 1305 205.92 * 109:33 29.75 * 13

* Best value for the (isolated) metric

factor (independent variable) to which two treatment instances are associated: the AGD method and the RUP generic process. More, a set of experimental observations (dependent variables) is composed of 6 principal metrics related with the process and the software product. These metrics are shown in table 2. The metrics allow researchers quantifying the performance of each development team. The overall experiment technically consisted in a series of simultaneous univariate experiments: an independent experiment was performed for each development aspect or metric that have to be observed. All experiments share a common experimental design and a common raw data sample. To collect data from two different environments, we had to build a common experiment layer that includes 10 data-capture forms (see table 1), five folders to allocate the results (see Fig. 3), and the procedure of experimentation (see section 3.3). The research hypothesis (see section 3) is translated into a statistically tractable form, called statistical hypotheses [20], as a pair: Null hypothesis, and Alternate hypothesis. The Null hypothesis H0 is that the means of the distributions of the metric in turn, for groups working with RUP, and AGD are equal (A = R), and the alternative hypothesis HA is that the means are different, and one of them is larger (A R and (A < R or A > R)). The statistical test used in the study is the two-sample t test [7] [20], assuming that two populations are governed by normal probability distributions, and that standard deviations of both populations are equal 1 = 2. We also use a nonparametric test for small samples, requiring neither of those assumptions, the Wilconox rank sum test. The Wilconox test is applied to obtain the threshold value or p-value of (associated with type I error). p-value is the value of that will barely cause the Null hypothesis H0 to be rejected.

and used at the equipment where the product will run at daily operation. This situation in the experiment context takes place at the acceptance test, after teams hand in their products to the client, and prepare the working environment to run the application. Metric M1_A: defects found in acceptance test, at row 8 of table 2 summarize the acceptance test procedure done by the client and researchers. Applying with these data the hypothesis test about two means, the Null hypothesis (A = R) is rejected with a 93.5% level of confidence, and a p-value of 0.167. This denotes that there is a truly significant effect, other than chance alone, and A < R, therefore client evaluates better quality for the product developed with AGD (A = 8 Defects). The same validation can be drawn from table 3, at rows 2, using the Acceptance Defect Density that computes defects found in one thousand function points. The Null hypothesis (A = R) is rejected with a level of confidence equal to 81.8%, and a p-value equal to 0.333. This denote that there is a truly significant effect, and A < R. With A = 60.53 Defects/1KFP. Other statistical hypothesis is that a team working with AGD method would deliver less LOCs by function point than a team working with RUP. The Effectiveness ratio, shown at row 5 of table 3, relates lines of code to function points, and concerns directly with quality, of the software product. So that effectiveness: a) has to be optimized by means of coding only the necessary amount of LOCs, and b) measures productivity giving a quality context to software development. Applying the hypothesis test about two means, to effectiveness in row 5, the Null hypothesis (A = R) is not rejected with a 39% level of confidence, and a p-value of 0.667. This denotes that there is no certainty of a difference in effectiveness between using AGD or using RUP, and suggests to do more experimentation. Groups working with RUP performed with similar effectiveness as groups working with AGD.

4. RESULTS: SMALLER ACCEPTANCE TEST DEFECT DENSITY

Our research hypothesis of Higher quality software products is translated to a set of statistical hypothesis, where each compares means of distributions of one metric. One statistical hypothesis that validates an improvement in software development, when using AGD method is: A team working with AGD method releases software products with smaller defect density, at acceptance test, than a team working with RUP. This corresponds to the alternate hypothesis HA for metric M1_A. A user of a software product perceives its quality mainly by the number of defects she or he observes when the product is installed

5. OTHER RESULTS AND DISCUSSION

From the Product functionality metric M7_T, and the Omissions M3_A metric identified by client, we determine the amount of function points satisfied by team RT(2) (shown as row 3 in Table 3). They are determined as follows: FP_satisfied (RT(2)) = AFP M3_A(RT(2)) = 202.3 29.75 = 172.55 From this result, we can estimate the deviation of the team from the originally planned functionality. This deviation is shown as row 4 of Table 3. The deviation quantifies the function points

Table 3 - Compound metrics computed to analyze results of experiment

Combined Metrics Acceptance Defect Density M1_A*1000/M2_CN (Defects/1K LOC) Acceptance Defect Density-FP (Defects/1K FP) M1_A*1000/M7_T FP_satisfied (FP) (AFP M3_A) FP_deviation (FP) (M7_T FP_satisfied) Effectiveness M2_CN/M7_T (LOCs/FP) * Best value for the (isolated) metric AT(1) 8.1 63.83 82.11 11.89 * 7.28 AT(2) 3.8 * 57.2 * 83.3 91.42 15.68 RT(1) 7.1 89.29 82.11 63.49 13.00 RT(2) 10.0 63.13 172.55 * 33.37 6.59 *

provided by this team that were not initially required: FP_deviation (RT(2)) = M7_T (TR(2)) FP_satisfied (RT(2)) = 205.92 172.55 = 33.37 The effort to provide this 33.37 function points could be used to develop the omissions identified by client M3_A = 29.75 function points (see Table 1), guiding the process improvement activities to prevent deviations. For the FP_deviation (row 4 of Table 3), the Null hypothesis (A = R) is not rejected in regards to a 47% level of confidence and a p-value of 0.667. This denotes that there is no certainty of the existence of an FP_deviation difference using AGD or RUP, and this suggests us to perform more experimentation. Acting with AGD, groups considered with a higher level of attention the characteristics of the work products while setting the strategy and during preliminary planification of their actions. Then, both teams working with AGD decided to reduce the amount of functionality they have to work on. The AT(1) team decided to reduce the scope, and only developed the originally asked functionality concerning this scope. So, they had the smallest FP_deviation. While AT(2) decided to reduce the scope too, but delivered more features (alternative queries, displays, and options), and thus they had the highest FP_deviation. The teams that worked with RUP were mainly concerned with their development process, and their work breakdown. The RT(2) team divided the construction effort identifying three principal options of the main display, and assigned one option to each of their three members. They performed very good work and obtained the best functionality (see M7_T, in table 2), but all their three interfaces were different. Offering different interfaces the application becomes more complex to use, increasing the learning effort. From the questionnaires, answered after experimentation phase (see section 3.1), we noticed that the experiment layer provided such a well adapted environment to users working with RUP an AGD that quite fulfill the initial requirements. The experiment testbed includes capture forms and predefined documentation folders that improve performance of teams working with RUP. It was also evident the AGD method and the experiment design have to be improved, and by this way we obtained useful evaluations and feedbacks from users. Concerning the group-work organization, the teams working with RUP assigned different roles during Inception and Elaboration phases. These roles are directly assigned analyzing the professional domains, e.g. Architect, Designer. At Construction and Transition phases, they had only two roles: Developers, and Project Manager. In contrast, using AGD for all working cycles, the team members acted using different suited team-oriented roles, e.g. team leader, planning manager, support manager, implementer, etc.

development defaults, we proposed the AGD (Architectural and Group Development) software development method that is mainly guided by the product architecture, and its process is highly guided by group dynamic. In order to show the AGD method improves some aspects of software development, we performed some experiments evaluating both the RUP process and the AGD method. The results confirm the AGD method allows developers producing software products with smaller defect density at acceptance test. We gave main attention to the work products and to the synergy provided by the four AGD group-work modes: Team, Collaborative, Cooperative, and Control. However, we have less confidence in the obtained differences evaluating the other metrics observing the groups that respectively worked with RUP and AGD. This fact suggests the need for more experimentations adapting/extending the prototype with remarks and suggestions produced by participants: 1) Potential possibility of the used methodologies? 2) A dedicated experiment layer as required by ongoing predefined weekly session? 3) Possibility to pre-allocate a predefined weekly time block for revision of work products? And 4) Documentation improvement of the experiment layer? From this experimentation, some main propositions for the AGD method evolution had been identified 1) Inclusion of the contents of the output document folders in examples, 2) Improvement of the method documentation with better presentation of ProductSet and group-work modes, and 3) Evaluation of the capture forms, analyzing the experiment results, and modification of the forms as required. This experience enforces and confirms our initial belief to apply the AGD method to other projects in order 1) to get a better understanding, 2) to diffuse the method to practitioners, and 3) to evaluate more specific method elements in order to improve them.

7. ACKNOWLEDGMENTS
This research has been developed with financial support provided by CONACyT, Mxico. The experiments were conducted within

regular academic courses given by the Department of Computer Sciences at CENIDET (Mexican Center for Research and Development of Technology). We would thank for their participation Mr. Mario Moreno, librarian of the Institute, Prof. Ren Santaolaya, Prof. Mario Guilln, and their enthusiastic students.

8. REFERENCES
[1] S. W. Ambler, Agile Modeling: Effective Practices for Extreme Programming and the Unified Process, John Wiley & Sons Inc., New York, 2002.

6. CONCLUSIONS AND FUTURE WORK

Software development environments are evolving towards powerful and useful software process, allowing users to rapidly produce interesting and usable results. In addition, software products that support group work are becoming more complex, and especially they have to provide dedicated supports for different application areas that require more functionality. In order to provide efficient solutions for the inherent software

[2] A. J. Albrecht, Measuring Application Development Productivity, In Proc. of the IBM Application Development Symposium, GUIDE Int and Share Inc., pp 83-92, Monterrey, USA, October 14-17, 1979. [3] R. Axelrod, The Evolution of Cooperation, Basic Books Press, 1984. [4] R. Axelrod, The Complexity of Cooperation, Princeton University Press, 1997.

[5] G. Antoniol, F. Catzolari, L. Cristoforetti, R. Fiutem, G Caldiera, Adapting Function Points to Object Oriented Information Systems, In Proc. Advanced Information Systems Engineering: CAISE'98, Lecture Notes in Computer Science, LNCS Vol. 1413, pp. 59-74, 1998. [6] V. R. Basili, Software Modeling and Measurement: The Goal Question Metric Paradigm, Computer Science Technical Report Series, CS-TR-2956 (UMIACS-TR-92-96), University of Maryland, College Park, MD, September 1992. [7] J. W. Barnes, Statistical Analysis for Engineers and Scientists: A Computer-Based Approach, McGraw-Hill Inc., pp. 199-200 and 341-343, New-York, 1994. [8] M. Gonzlez Garca, R. Jacinto Montes, A. M. Martnez Enrquez, CWSLR Model for CSCW Information Systems, In Proc. PDPTA99, pp. 53-59, June 28 July 1, 1999. [9] M. Gonzlez Garca, R. Jacinto Montes, A. M. Martnez Enrquez, CWSLR Model Used to Synthesize a Software Development Environment and an Application Design Tool, In Proc. PDPTA2000, June 2000. [10] M. Gonzlez Garca A. M. Martnez Enrquez, Integration of Concepts in a Software Development Environment, In Proc. of The 2002 International Multiconferences in Computer Science, SERP02, pp. 184 190, June 24-27, 2002. [11] W. Hesse, Dinosaur Meets Archaeopteryx? - Seven Theses on Rationals Unified Process (RUP), In Proc. CAiSE/IFIP 8.1 Int. Workshop on Evaluation of Modeling Methods in System Analysis and Design (EMMSAD01), Interlaken 2001. [12] W. Hesse; Dinosaur meets Archaeopteryx? or: Is there an alternative for Rational's Unified Process?, Software and System Modeling Vol. 2, No. 4, pp. 240-247, 2003. [13] W. Humphrey, Introduction to Team Software Process, Addison-Wesley Professional, August 1999. [14] IEEE Std. 1028 - 98; IEEE Standard for software reviews; IEEE Computer Society, Software Engineering Standards Committee, March 1998. [15] IEEE/EIA 12207 (0-1996, 1-1997, 2-1997) Ind. Imp. of International Standard ISO/IEC 12207: 1995 (Standard for Information Technology Software life cycle processes); IEEE, 1998. [16] Standard Glossary of Software Engineering Technology, ANSI/IEEE Std. 610.12, 1990. [17] ISO/IEC 10746-1, 2, 3, 4 | ITU-T Recommendation X.901, X.902, X.903, X.904. "Open Distributed Processing Reference Model". OMG, 1995-98. [18] ISO/IEC; ISO/IEC TR 15504: Information Technology Software Process Assessment (parts 1- 9); ISO/IEC, 1998 (part 5 was published in 1999). [19] I. Jacobson, G. Booch, and J. Rumbaugh. The Unified Software Development Process, Addison Wesley, 1999.

[20] E. Kreyszig, Introductory Mathematical Statistics: Principles and Methods, John Wiley & Sons 1970, Ch. 13, 20, pp. 219-228, 234-239, 403-405, Spanish version by Limusa, 1978. [21] P. Kroll, Planning and estimating a RUP project using IBM Rational SUMMIT Ascendant, IBM Rational, 11 May 2004. [22] J. T. Nosek, The Case for Collaborative Programming, Communications of the ACM, Vol. 41, No. 3, pp. 105-108, 1998. [23] OMG-MDA Model Driven Architecture Guide Version 1.0, omg/2003-05-01, 1 May 2003. [24] G. Orrand, T. E. Reeves, Function Point Counting: One Programs Experience, Journal of Systems and Software Vol. 53, N.3, pp.239-244, 2000. [25] M. C. Paulk, C. V. Weber, B. Curtis, and M. B. Chrissis, Capability Maturity Model: The Guidelines for Improving the Software Process, Addison Wesley Professional, The SEI Series in Software Engineering, June 1995. [26] RUP implementation guide Tart I: Recommended strategy and typical issues and risks, The Rational Edge, 19 August 2003. [27] RUP implementation guide Tart II: Best Practices and Key Lessons Learned from RUP implementation projects, The Rational Edge, November 2003. [28] M .Shaw, D. Garlan, R. Allen, D. Klein, J. Ockerbloom, C. Scott, and M. Schumaker, Candidate Model Problems in Software Architecture, Draft Technical Report 15-675, CSD CMU, 1995. [29] A. P. Snow, M. Keil, The Challenges of Accurate Project Status, In Proc. 34 Annual Hawaii International Conference on System Sciences (HICSS-34), Vol. 8, pp. 3133-3142, 2001 [30] A. Abraham et al. (executive editors), Pierre Bourque et al (editors), Guide to the Software Engineering Body of Knowledge, IEEE Computer Society Professional Practices Committee, 2004. [31] P. D. Stotts, L. Williams, N. Nagappan, P. Baheti, D. Jen, A. Jackson, Virtual Teaming: Experiments and Experiences with Distributed Pair Programming, XP/Agile Universe, pp. 129-141, 2003. [32] L. A. Williams, R. R. Kessler, All I Ever Needed to Know About Pair Programming I learned in Kinder-garten, Communications of ACM, Vol. 43, No. 5, pp. 108-114, 2000. [33] P. R. Wyrick, Unified Software Project Management Using Rational ProjectConsole to Collect Microsoft Project Metrics, Rational Software White paper, 17 May 2001. [34] P. R. Wyrick, B. Gilbert, and D. West, Unified Software Project Management Rational Project Console Sample Metrics, Rational Software White paper, 2001.