Software

Q : If you have to write a 10,000 line program in C to solve a problem, how long will it take? Answers: generally range from 2-4 months Let us analyze the productivity

Productivity = output/input resources In SW output is considered as LOC Input resources is effort - person months; overhead cost modeled in rate for person month Though not perfect, some productivity measure is needed, as project has to keep it high

Software

The productivity is 2.5-5 KLOC/PM Q: What is the productivity in a typical commercial SW organization ? A: Between 100 to 1000 LOC/PM Q: Why is it low, when your productivity is so high? (people like you work in the industry) A: What the student is building and what the industry builds are two different things
In a univ a student system is built while the commercial org builds industrial strength sw What is the difference between a student program and industrial strength sw for the same problem? Software (IEEE): collection of programs, procedures, rules, and associated documentation and data

Software

Software

Student Developer is the user

bugs are tolerable UI not important No documentation

Industrial Strength

Others are the users

bugs not tolerated UI v. imp. issue Documents needed for the user as well as for the organization and the project

Software

Student SW not in critical use Reliability, robustness not important No investment Dont care about portability Industrial Strength Supports important functions / business Reliability , robustness are very important Heavy investment Portability is a key issue here

Industrial strength software

Student programs for a problem & industrial strength software are two different things Key difference is in quality (including usability, reliability, portability, etc.) Brooks thumb-rule: Industrial strength sw costs 10 time more than student sw In this course, software means industrial strength software This software has some characteristics Let us look at costs involved

Is Expensive

Productivity = Appx 1000 LOC/PM Cost = $3K to $10K/PM Cost per LOC = $5 to $15 I.e, each line of delivered code costs many $s Cost = $100K to $2.25Million Can easily run on $10K-$20K hardware So HW costs in an IT solution are small as compared to SW costs.

A simple application for a business may have 20KLOC to 50KLOC

Requires tight Schedule

Business requirements today demand short delivery times for software In the past, software products have often failed to be completed in time Along with cost, cycle time is a fundamental driving force An industrial strength software project is driven by cost and schedule Both can be modeled by productivity, measured in terms of output per unit effort (e.g. LOC per person month)

Productivity for cost and schedule

Higher productivity leads to lower cost Higher productivity leads to lower cycle time

Hence, for projects (to deliver software), quality and productivity are basic drivers

Quality

Along with productivity, quality is the other major driving factor Developing high Q sw is a basic goal Quality of sw is harder to define

Quality ISO standard Quality ISO std

ISO std has six attributes

Functionality Reliability Usability Efficiency Maintainability Portability

Quality

Multiple dimensions mean that not easy to reduce Q to a single number Concept of Q is project specific

For some reliability is most important For others usability may be more important

Reliability is generally considered the main Q criterion Reliability = Probability of failure

Quality

hard to measure approximated by no. of defects in software Quality = No. of defects delivered / Size

To normalize Quality = Defect density

Defects delivered - approximated with no. of defects found in operation Current practices: less than 1 def/KLOC What is a defect? Project specific!
Once sw delivered, it enters the maintenance phase, in which

Quality Maintainability

Residual errors are fixed this is corrective maintenance Upgrades and environment changes are done this is adaptive maintenance

Maintenance can consume more effort than development over the life of the software (can even be 20:80 ratio!) Hence maintainability is another quality attribute of great interest

Quality and Productivity

Hence, quality and productivity (Q&P) are the basic drivers in a sw project The aim of most methodologies is to deliver software with a high Q&P Besides the need to achieve high Q&P there are some other needs Only constant in business is change! Requirements change, even while the project is in progress In a project, up to 40% of development effort may go in implementing changes

Change

Practices used for developing software must accommodate change Most industrial strength software tend to be large and complex Methods for solving small problems do not often scale up for large problems Two clear dimensions in a project

Scale

For small, both can be done informally, for large both have to be formalized

engineering project management

Scale Scale

An illustration of issue of scale is counting the number of people in a room vs taking a census

Both are counting problems Methods used in first not useful for census For large scale counting problem, must use different techniques and models Management will become critical

Scale: Examples Scale

As industry strength software tends to be large, hence methods used for building these must be able to scale up For much of the discussion, we will high Q&P as the basic objective
The problem domain for SE is industrial strength software SE aims to provide methods for systematically developing (industrial strength) software

Summary

Besides developing software the goal is to achieve high quality and productivity (Q&P) Methods used must accommodate changes, and must be able to handle large problems

Software Process Software Engineering

We have specified the problem domain industrial strength software

Software engineering is defined as the systematic approach for development of (industrial strength) software Q&P is an essential goal Q&P depends on people, process, and technology

Besides delivering the software, cost, quality, and schedule are drivers

Process, People, Technology

Software Process

Processes help people become more productive and create fewer errors Tools help people execute some tasks in the process more efficiently and effectively So, process forms the core

Process is distinct from product products are outcomes of executing a process on a project SW Engg. focuses on process Premise: Proper processes will help achieve project objectives of high QP

The software Development Problem Project and Process

A software project is one instance of the development problem

Development process takes the project from user needs to software There are other goals of cost schedule and quality, besides delivering software Need other processes
Process: A sequence of steps performed to achieve some goal Software Process: The sequence of steps performed to produce software with high quality, within budget and schedule Many types of activities performed by diff people in a software project Better to view software process as comprising of many component processes Two major processes

Software Process

Component Software Processes

Development focuses on development and quality steps needed to engineer the software Project management focuses on planning and controlling the development process

Development process is the heart of software process; other processes revolve around it These are executed by different people

developers execute engg. Process project manager executes the mgmt proces

Component Processes

Other processes

Configuration management process: manages the evolution of artifacts Change management process: how changes are incorporated Process management process: management of processes themselves Inspection process: How inspections are conducted on artifacts

Process Specification

Process is generally a set of phases

Each phase performs a well defined task and generally produces an output Intermediate outputs work products At top level, typically few phases in a process How to perform a particular phase methodologies have been proposed ETVX approach to specify a step

ETVX Specification

Entry criteria: what conditions must be satisfied for initiating this phase Task: what is to be done in this phase Verification: the checks done on the outputs of this phase eXit criteria: when can this phase be considered done successfully

A phase also produces info for mgmt

ETVX approach Development Process and Process Models Software Project

Project to build a sw system within cost and schedule and with high quality which satisfies the customer Suitable process needed to reach goals Process should not just help produce the software but help achieve the highest Q&P For a project, the projects process to be followed is specified during planning A process model specifies a general process that is optimal for a class of problems A project may select its process using one of the process models
A set of phases and each phase being a sequence of steps

Project s process and Process Models

Development Process

Sequence of steps for a phase - methodologies for that phase. Why have phases

To employ divide and conquer each phase handles a different part of the problem helps in continuous validation

Development Process

Commonly has these activities: Requirements analysis, architecture, design, coding, testing, delivery Different models perform them in different manner A process model specifies a general process, usually as a set of stages This model will be suitable for a class of projects I.e. a model provides generic structure of the process that can be followed by some projects to achieve their goals Linear sequence of stages/phases Requirements HLD DD Code Test Deploy A phase starts only when the previous has completed; no feedback The phases partition the project, each addressing a separate concern

Process Models

Waterfall Model

Waterfall

Linear ordering implies each phase should have some output The output must be validated/certified Outputs of earlier phases: work products Common outputs of a waterfall: SRS, project plan, design docs, test plan and reports, final code, supporting docs

Waterfall Advantages

Conceptually simple, cleanly divides the problem into distinct phases that can be performed independently Natural approach for problem solving Easy to administer in a contractual setup each phase is a milestone Assumes that requirements can be specified and frozen early May fix hardware and other technologies too early Follows the big bang approach all or nothing delivery; too risky Very document oriented, requiring docs at the end of each phase Has been used widely Well suited for projects where requirements can be understood easily and technology decisions are easy I.e. for familiar type of projects it still may be the most optimum Prototyping addresses the requirement specification limitation of waterfall Instead of freezing requirements only by discussions, a prototype is built to understand the requirements Helps alleviate the requirements risk A small waterfall model replaces the requirements stage

Waterfall disadvantages

Waterfall Usage

Prototyping

Prototyping

Prototyping

Development of prototype

Starts with initial requirements Only key features which need better understanding are included in prototype No point in including those features that are well understood Feedback from users taken to improve the understanding of the requirements

Prototyping

Cost can be kept low

Prototyping

Build only features needing clarification quick and dirty quality not important, scripting etc can be used Things like exception handling, recovery, standards are omitted Cost can be a few % of the total Learning in prototype building will help in building, besides improved requirements

Advantages: req will be more stable, req frozen later, experience helps in the main development Disadvantages: Potential hit on cost and schedule

Applicability: When req are hard to elicit and confidence in reqs is low; i.e. where reqs are not well understood
Counters the all or nothing drawback of the waterfall model Combines benefit of prototyping and waterfall Develop and deliver software in increments Each increment is complete in itself Can be viewed as a sequence of waterfalls Feedback from one iteration is used in the future iterations

Iterative Development

Iterative Enhancement

Iterative Development

Products almost always follow it Used commonly in customized development also

Businesses want quick response for sw Cannot afford the risk of all-or-nothing

Newer approaches like XP, Agile, all rely on iterative development Benefits: Get-as-you-pay, feedback for improvement,

Iterative Development..

Drawbacks: Architecture/design may not be optimal, rework may increase, total cost may be more Applicability: where response time is important, risk of long projects cannot be taken, all req not known

Phases in a Project

Phases in a project

Inception phase: ends with Lifecycle Objectives milestone; vision and high level capability of system defined Elaboration phase: Lifecycle architecture milestone; most requirements defined and architecture designed Construction phase: Initial operational capability milestone Transition phase: Product release; transition product from development to production

Phases and Milestones

Execution of phases

Each phase itself can be done in multiple iterations, each iteration having an external/internal customer Generally construction has multiple iterations; elaboration can also be meaningfully done in multiple iterations Engineering tasks are called core process workflows

Core workflows and phases

These sub processes correspond to tasks of requirements, design, implementation, testing, proj mgmt, etc Many sub processes may be active in a phase, the volume of activity generally differs depending on the project

Sub processes and phases

Summary waterfall

Summary Prototyping

Summary Iterative

Summary

Process is a means to achieve project objectives of high QP Process models define generic process, which can form basis of project process Process typically has stages, each stage focusing on an identifiable task Many models for development process have been proposed
Development process models discussed

Summary

Waterfall Prototyping Iterative RUP Timeboxing Agile or XP

Project Management Process

Each has its strengths and weaknesses and will work well for some types of projects

Background

PM Process Phases

Development process divides development into phases and activities To execute it efficiently, must allocate resources, manage them, monitor progress, take corrective actions, These are all part of the PM process Hence, PM process is an essential part of executing a project There are three broad phases
Planning Monitoring and control Termination analysis

Planning is a key activity that produces a plan, which forms the basis of monitoring

Planning

Done before project begins Key tasks

Cost and schedule estimation Staffing Monitoring and risk mgmt plans Quality assurance plans Etc.

Will discuss planning in detail later Lasts for the duration of the project and covers the development process

Monitoring and control

Monitors all key parameters like cost, schedule, risks Takes corrective actions when needed Needs information on the dev process provided by metrics

Termination Analysis

Termination analysis is performed when the development process is over Basic purpose: to analyze the perf of the process, and identify lessons learned Also called postmortem analysis

Relationship with Dev Process

Summary

Process has a major impact on the quality and productivity Different processes at work in a software project We have focused on development process and project management process Process models are general process structures, that work well for some types of problems A project should select a process model that is best suited for it (and tailor it to meet its requirements)

Software Requirements Analysis and Specification

Background

Problem of scale is a key issue for SE For small scale, understand and specifying requirements is easy For large problem - very hard; probably the hardest, most problematic and error prone Input : user needs in minds of people Output : precise statement of what the future system will do

Background..

Identifying and specifying req necessarily involves people interaction Cannot be automated Requirement (IEEE)= A condition or capability that must be possessed by a system Req. phase ends with a software requirements specification (SRS) document SRS specifies what the proposed system should do Requirements understanding is hard

Background..

Visualizing a future system is difficult Capability of the future system not clear, hence needs not clear Requirements change with time

Essential to do a proper analysis and specification of requirements

Need for SRS SRS establishes basis of agreement between the user and the supplier.

Users needs have to be satisfied, but user may not understand software Developers will develop the system, but may not know about problem domain SRS is the medium to bridge the commn. gap and specify user needs in a manner both can understand

Need for SRS

Helps user understand his needs.

users do not always know their needs must analyze and understand the potential the goal is not just to automate a manual system, but also to add value through IT The req process helps clarify needs Clear understanding about what is expected. Validation - SW satisfies the SRS

SRS provides a reference for validation of the final product

Need for SRS

High quality SRS essential for high Quality SW

Requirement errors get manifested in final sw to satisfy the quality objective, must begin with high quality SRS

Requirements defects are not few

Need for SRS Good SRS reduces the development cost

25% of all defects in one case; 54% of all defects found after UT 80 defects in A7 that resulted in change requests 500 / 250 defects in previously approved SRS.

SRS errors are expensive to fix later Req. changes can cost a lot (up to 40%) Good SRS can minimize changes and errors Substantial savings; extra effort spent during req. saves multiple times that effort Cost of fixing errors in req. , design , coding , acceptance testing and operation are 2 , 5 , 15 , 50 , 150 person-months

An Example

Need for SRS

Example

After req. phase 65% req errs detected in design , 2% in coding, 30% in Acceptance testing, 3% during operation If 50 requirement errors are not removed in the req. phase, the total cost 32.5 *5 + 1*15 + 15*50 + 1.5*150 = 1152 hrs If 100 person-hours invested additionally in req to catch these 50 defects , then development cost could be reduced by 1152 person-hours. Net reduction in cost is 1052 person-hours

Requirements Process

Sequence of steps that need to be performed to convert user needs into SRS Process has to elicit needs and requirements and clearly specifies it Basic activities

problem or requirement analysis requirement specification validation

Analysis involves elicitation and is the hardest

Requirement process..

Requirements Process

Process is not linear, it is iterative and parallel Overlap between phases - some parts may be analyzed and specified Specification itself may help analysis Validation can show gaps that can lead to further analysis and spec
Focus of analysis is on understanding the desired systems and its requirements Divide and conquer is the basic strategy

decompose into small parts, understand each part and relation between parts organizing them is a key

Large volumes of information is generated

Techniques like data flow diagrams, object diagrams etc. used in the analysis

Requirements Process..

Transition from analysis to specs is hard

in specs, external behavior specified during analysis, structure and domain are understood analysis structures helps in specification, but the transition is not final methods of analysis are similar to that of design, but objective and scope different analysis deals with the problem domain, whereas design deals with solution domain

Problem Analysis

Aim: to gain an understanding of the needs, requirements, and constraints on the software Analysis involves

interviewing client and users reading manuals studying current systems helping client/users understand new possibilities Like becoming a consultant

Problem Analysis

Must understand the working of the organization , client and users

Some issues

Obtaining the necessary information Brainstorming: interacting with clients to establish desired properties Information organization, as large amount of info. gets collected Ensuring completeness Ensuring consistency Avoiding internal design

Problem Analysis

Interpersonal issues are important Communication skills are very important Basic principle: problem partition

Partition w.r.t what?

Object - OO analysis Function - structural analysis Events in the system event partitioning

Projection - get different views Will discuss few different analysis techniques What should be the characteristics of a good SRS? Some key ones are

Characteristics of an SRS

Complete Unambiguous Consistent Verifiable Ranked for importance and/or stability

Characteristics

Correctness

Completeness

Each requirement accurately represents some desired feature in the final system All desired features/characteristics specified Hardest to satisfy Completeness and correctness strongly related Each req has exactly one meaning Without this errors will creep in Important as natural languages often used

Unambiguous

Characteristics

Verifiability

There must exist a cost effective way of checking if sw satisfies requirements two requirements dont contradict each other Needed for prioritizing in construction To reduce risks due to changing requirements

Consistent

Ranked for importance/stability

Components of an SRS

What should an SRS contain ?

An SRS must specify requirements on

Clarifying this will help ensure completeness

Functionality Performance Design constraints External interfaces

Functional Requirements

Performance Requirements

Heart of the SRS document; this forms the bulk of the specs Specifies all the functionality that the system should support Outputs for the given inputs and the relationship between them All operations the system is to do Must specify behavior for invalid inputs too

All the performance constraints on the software system Generally on response time , throughput etc => dynamic Capacity requirements => static Must be in measurable terms (verifiability)

Design Constraints

Eg resp time should be xx 90% of the time

Factors in the client environment that restrict the choices Some such restrictions
Standard compliance and compatibility with other systems Hardware Limitations Reliability, fault tolerance, backup req. Security

External Interface

All interactions of the software with people, hardware, and sw User interface most important General requirements of friendliness should be avoided These should also be verifiable Language should support desired char of the SRS

Specification Language

Formal languages are precise and unambiguous but hard Natural languages mostly used, with some structure for the document Formal languages used for special features or in highly critical systems Structure of an SRS

Introduction

Purpose , the basic objective of the system Scope of what the system is to do , not to do Overview Product perspective Product functions User characteristics Assumptions Constraints

Overall description

Structure of an SRS

Specific requirements

Acceptable criteria

External interfaces Functional requirements Performance requirements Design constraints desirable to specify this up front.

This standardization of the SRS was done by IEEE.

Use Cases Approach for Functional Requirements

Traditional approach for fn specs specify each function Use cases is a newer technique for specifying behavior (functionality) I.e. focuses on functional specs only Though primarily for specification, can be used in analysis and elicitation Can be used to specify business or org behavior also, though we will focus on sw Well suited for interactive systems

Use Cases Basics

A use case captures a contract between a user and system about behavior Basically a textual form; diagrams are mostly to support Also useful in requirements elicitation as users like and understand the story telling form and react to it easily
Actor: a person or a system that interacts with the proposed system to achieve a goal

Basics..

Actor is a logical entity, so receiver and sender actors are different (even if the same person) Actors can be people or systems Primary actor: The main actor who initiates a UC

Eg. User of an ATM (goal: get money); data entry operator; (goal: Perform transaction)

UC is to satisfy his goals The actual execution may be done by a system or another person on behalf of the Primary actor

Basics..

Scenario: a set of actions performed to achieve a goal under some conditions

Actions specified as a sequence of steps A step is a logically complete action performed either by the actor or the system

Main success scenario when things go normally and the goal is achieved Alternate scenarios: When things go wrong and goals cannot be achieved

Basics..

A UC is a collection of many such scenarios A scenario may employ other use cases in a step I.e. a sub-goal of a UC goal may be performed by another UC I.e. UCs can be organized hierarchically

Basics

UCs specify functionality by describing interactions between actors and system Focuses on external behavior UCs are primarily textual

UC diagrams show UCs, actors, and dependencies They provide an overview

Story like description easy to understand by both users and analysts They do not form the complete SRS, only the functionality part

Requirements with Use Cases

UCs specify functional requirements Other req identified separately A complete SRS will contain the use cases plus the other requirements Note for system requirements it is important to identify UCs for which the system itself may be the actor UCs form a good medium for brainstorming and discussions Hence can be used in elicitation and problem analysis also UCs can be developed in a stepwise refinement manner

Developing Use Cases

Many levels possible, but four naturally emerge

Developing

Step 1: Identify actors and goals

Prepare an actor-goal list Provide a brief overview of the UC This defines the scope of the system Completeness can also be evaluated For each UC, expand main scenario This will provide the normal behavior of the system Can be reviewed to ensure that interests of all stakeholders and actors is met

Step 2: Specify main Success Scenarios

Developing

Step 3: Identify failure conditions

List possible failure conditions for UCs For each step, identify how it may fail This step uncovers special situations Perhaps the hardest part Specify system behavior for the failure conditions New business rules and actors may emerge

Step 4: Specify failure handling

Other Approaches to Analysis Data Flow Modeling

Widely used; focuses on functions performed in the system Views a system as a network of data transforms through which the data flows Uses data flow diagrams (DFDs) and functional decomposition in modeling The SSAD methodology uses DFD to organize information, and guide analysis A DFD shows flow of data through the system

Data flow diagrams

Views system as transforming inputs to outputs Transformation done through transforms DFD captures how transformation occurs from input to output as data moves through the transforms Not limited to software

Data flow diagrams

DFD

Transforms represented by named circles/bubbles Bubbles connected by arrows on which named data travels A rectangle represents a source or sink and is originator/consumer of data (often outside the system)

DFD Conventions

External files shown as labeled straight lines Need for multiple data flows by a process represented by * (means and) OR relationship represented by + All processes and arrows should be named Processes should represent transforms, arrows should represent some data

Data flow diagrams

Focus on what transforms happen , how they are done is not important Usually major inputs/outputs shown, minor are ignored in this modeling No loops , conditional thinking , DFD is NOT a control chart, no algorithmic design/thinking Sink/Source , external files

Drawing a DFD

If get stuck , reverse direction If control logic comes in , stop and restart Label each arrows and bubbles Make use of + & * Try drawing alternate DFDs Leveled DFDs : DFD of a system may be very large Can organize it hierarchically Start with a top level DFD with a few bubbles then draw DFD for each bubble Preserve I/O when exploding

Drawing a DFD for a system

Identify inputs, outputs, sources, sinks for the system Work your way consistently from inputs to outputs, and identify a few high-level transforms to capture full transformation If get stuck, reverse direction When high-level transforms defined, then refine each transform with more detailed transformations

Drawing a DFD for a system..

Never show control logic; if thinking in terms of loops/decisions, stop & restart Label each arrows and bubbles; carefully identify inputs and outputs of each transform Make use of + & * Try drawing alternate DFDs
DFD of a system may be very large Can organize it hierarchically Start with a top level DFD with a few bubbles then draw DFD for each bubble Preserve I/O when exploding a bubble so consistency preserved Makes drawing the leveled DFD a top-down refinement process, and allows modeling of large and complex systems

Leveled DFDs

Data Dictionary

In a DFD arrows are labeled with data items Data dictionary defines data flows in a DFD Shows structure of data; structure becomes more visible when exploding Can use regular expressions to express the structure of data

Data Dictionary Example

For the timesheet DFD

Weekly_timesheet employee_name + id + [regular_hrs + overtime_hrs]* Pay_rate = [hourly | daily | weekly] + dollar_amt Employee_name = last + first + middle Id = digit + digit + digit + digit

DFD drawing common errors

Unlabeled data flows Missing data flows Extraneous data flows Consistency not maintained during refinement Missing processes Too detailed or too abstract Contains some control information Prototyping is another approach for problem analysis Discussed it earlier with process leads to prototyping process model
Lot of room for misunderstanding Errors possible Expensive to fix req defects later Must try to remove most errors in SRS Most common errors

Prototyping

Requirements Validation

Omission

- 30%

Inconsistency Incorrect fact Ambiguity

- 10-30% - 10-30% - 5 -20%

Requirements Review

SRS reviewed by a group of people Group: author, client, user, dev team rep. Must include client and a user Process standard inspection process Effectiveness - can catch 40-80% of req. errors Having a good quality SRS is essential for Q&P The req. phase has 3 major sub phases

Summary

analysis , specification and validation

Analysis

for problem understanding and modeling Methods used: SSAD, OOA , Prototyping

Key properties of an SRS: correctness, completeness, consistency,unambiguousness

Summary..

Specification

must contain functionality , performance , interfaces and design constraints Mostly natural languages used

Use Cases is a method to specify the functionality; also useful for analysis Validation - through reviews