Sunteți pe pagina 1din 94

An Introduction to Testing Life Cycle Process

Contents
Testing Life Cycle Overview Test Strategy Test Planning Test Case Design Test Execution Testing Types Testing Tools Overview

      

Testing Life Cycle Overview


       Testing Life Cycle Overview Test Strategy Test Planning Test Case Design Test Execution Testing Types Testing Tools Overview

Testing Life Cycle Overview


Normally, testing of any Large Systems will be in TWO parts. The functional verification and validation against the Requirement Spec and Performance evaluation against the indicated requirements Testing activity is involved right from the beginning of the project V&V Process model to successfully deliver a project on time This model is followed for a complete project life cycle model

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

Testing Life Cycle - V&V Process Model


Business requirements, Solution Architecture Master Test Strategy Release Acceptance Tests & Certification

Project Planning

Test Planning

Integrated Solution

Performance Testing, Business Cycle testing

HLD/LLD for each application involved Interface design

Strategy for individual applications, Test case design

Application Integration

Incremental Integration Testing

Solution Mapping/ Development/ Customization

System Testing (Application level Testing)

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

Testing Life Cycle - V&V Process Concept


The "V &V " concept relates the build components of development to the test components that occur during that build phase. Testing related activities during Requirement phase
Creation and finalization of testing templates Creation of over-all Test Plan and Test Strategy Capturing Acceptance criteria and preparation of Acceptance Test Plan Capturing Performance criteria of the software requirements
 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

Testing Life Cycle - V&V Process Concept


Testing activities in System Testing phase
System test is done for validating the product with respect to client requirements Testing can be in multiple rounds Defect found during system test should be logged into Defect Tracking System for the purpose of tracking. Test logs and defects are captured and maintained.  Testing Life Cycle Review of all the test documents Overview
 Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

Testing Life Cycle - V&V Process Concept


Testing activities in Design phase
Develop Test cases to ensure that product is on par with Requirement Specification document. Verify Test Cases & test scripts by peer reviews. Preparation of traceability matrix from system requirements
 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

Testing Life Cycle - V&V Process Concept


Testing activities in Integration Testing Phase
This testing is conducted in parallel with integration of various applications (or components) Testing the product with its external and internal interfaces without using drivers and stubs. Incremental approach while integrating the interfaces.
 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

Testing Life Cycle - V&V Process Concept


Performance Testing This is done to validate the performance criteria of the product / application. This is non-functional testing Business Cycle testing this refers to end to end testing of real life-like business scenarios. Testing activities during Release phase
Acceptance testing is conducted at the customer location. Resolves all defects reported by the customer during Acceptance Testing Conduct Root Cause Analysis (RCA) for those defects reported by customer during acceptance testing
 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

Testing Life Cycle - Team Structure


An effective testing team includes a mixture of members who has Testing expertise Tools expertise Database expertise Domain/Technology expertise

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

Testing Life Cycle - Team Structure (Contd)


The testing team must be properly structured, with defined roles and responsibilities that allow the testers to perform their functions with minimal overlap. There should not be any uncertainty regarding which team member should perform which duties. The test manager will be facilitating any resources required for the testing team.

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

Testing Life Cycle - Roles and Responsibilities


Clear Communication protocol should be defined with in the testing team to ensure proper understanding of roles and responsibilities. The roles chart should contain both on-site and off-shore team members.
 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

Testing Life Cycle - Roles and Responsibilities


Test Manager
Single point contact between onsite and offshore team Prepare the project plan Test Management Test Planning Interact with onsite lead, Client QA manager Team management Work allocation to the team Test coverage analysis

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

Testing Life Cycle - Roles and Responsibilities


Test Manager cont..
Co-ordination with onsite for issue resolution. Monitoring the deliverables Verify readiness of the product for release through release review Obtain customer acceptance on the deliverables Performing risk analysis when required Reviews and status reporting Authorize intermediate deliverables and patch releases to customer.

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

Testing Life Cycle - Roles and Responsibilities


Test Lead
Resolves technical issues for the product group Provides direction to the team members Performs activities for the respective product group Review and Approve of Test Plan / Test cases Review Test Script / Code Approve completion of Integration testing Conduct System / Regression tests Ensure tests are conducted as per plan Reports status to the Offshore Test Manager

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

Testing Life Cycle - Roles and Responsibilities


Test Engineer
Development of Test cases and Scripts Test Execution Result capturing and analysing Defect Reporting and Status reporting
 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

Testing Life Cycle - Term Definitions


Testing: It is a process of executing software in a controlled manner, in order to answer the question Does the software behave as specified? Verification: Says Are we building the product right? All QC activities throughout the life cycle that ensure that interim deliverables meet their input specification. Validation: Says Are we building the right product? The test phase of the life cycle which assures that the end product meets the users needs. Test Case: An input operation and the corresponding expected output in order to test a small unit of work. Test Script: A logical group of test cases which, when taken together, test a particular function or unit of a system

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

Test Bed: A group of test scripts which, when taken together, test all functions of an entire system Audit: An inspection/assessment activity that verifies compliance with plans, policies and procedures Baseline: A quantitative measure of the current level of performance Bug: A catch all term for all software defects or errors Certification: Acceptance of a software by an authorized agent or its validity been demonstrated to an authorized agent Cost of Quality (COQ): Money spent above and beyond expected production costs, this involves prevention, appraisal, internal & external failure costs Cyclomatic Complexity: The number of decision statements plus one.

Testing Life Cycle - Term Definitions (Contd.)

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

Defect: It is A deviation from specification or standard or Any thing that causes customer dissatisfaction. Desk Check: Verification technique conducted by the author of the artifact to verify the completeness Dynamic Assertion: A dynamic analysis technique that inserts into the program code assertions about the relationship between program variables Force field Analysis: A group technique used to identify both driving and restraining forces that influence a current situation Inspection: A formal assessment of a work product conducted by one or more qualified independent reviewers to detect defects, violation of development standards, etc. Inspection identifies defects but does not attempt to correct them.

Testing Life Cycle - Term Definitions (Contd.)

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

Testing Life Cycle - Term Definitions (Contd.)


Instrumentation: The insertion of the additional code into a program to collect information about a program behavior during program execution. Life Cycle Testing: The process of verifying the consistency, completeness and correctness of software at each phases of the development life cycle. Phase Containment: Method of control put in place within each stage of the development process to promote error identification and resolution so that defects are not propagated downstream to subsequent stages of the development process Productivity: The ratio of the output of a process to the input

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

Quality Assurance: The set of activities (including facilitation, training, measurement, and analysis) needed to provide adequate confidence that process are established continuously improved to produce products that meet specifications and are fit for use. Quality Control: The process by which product quality is compared and detected w.r.t requirements and other relevant specifications, focus is in detection and removal Test Driver: A program that directs the execution of another program against a collection of test data sets Test item: A software item that is an object of testing User: The customer that actually uses the product received

Testing Life Cycle - Term Definitions (Contd.)

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

Test Strategy
       Testing Life Cycle Overview Test Strategy Test Planning Test Case Designing Test Execution Testing Types Testing Tools Overview

Test Strategy
Test strategy is statement of overall approach of testing to meet the business and test objectives. It is a plan level document and has to be prepared in the requirement stage of the project. It identifies the methods, techniques and tools to be used for testing . It can be a project or an organization specific. Developing a test strategy which effectively meets the needs of the organization/project is critical to the success of the software development An effective strategy has to meet the project and business objectives Defining the strategy upfront before the actual testing helps in planning the test activities

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

Elements of Test Strategy


A test strategy will typically cover the following aspects
Definition of test objective Strategy to meet the specified objective Overall testing approach Test Environment Test Automation requirements Metric Plan Risk Identification, Mitigation and Contingency plan Details of Tools usage Specific Document templates used in testing

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

Test Strategy - Project/Business Objectives


All the business and test objectives are identified and test strategy shall be defined in order to meet those objectives. As far as possible all the test objectives to be met are defined in quantitative terms in the test strategy. Examples of test objectives can be like 90 % of code coverage, finding at least a minimum of 100 defects etc.

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

Test Strategy - Test Approach


Test approach will be based on the objectives set for testing Test approach will detail the way the testing to be carried out Types of testing to be done viz Unit, Integration and system testing The method of testing viz Blackbox, Whitebox etc., Details of any automated testing to be done

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

Test Strategy - Test Environment


All the Hardware and Software requirements for carrying out testing shall be identified in detail. Any specific tools required for testing will also be identified If the testing is going to be done remotely, then it has to be considered during estimation

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

Test Strategy - Metric Plan


Metric plan will contain the measures that will indicate the effectiveness of testing . All the metrics to be tracked for testing like defect density ,residual defect density ,code coverage etc are identified in test strategy . The objectives set for testing are also measured in the metric plan.

Note: More details are covered in Test Execution section

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

Test Strategy - Risks


Risk analysis should carried out for testing phase The risk identification will be accomplished by identifying causes-and-effects or effects-and-causes The identified Risks are classified into to Internal and External Risks. The internal risks are things that the test team can control or influence. The external risks are things beyond the control or influence of the test team Once Risks are identified and classified, the following activities will be carried out
Identify the probability of occurrence Identify the impact areas if the risk were to occur Risk mitigation plan how avoid this risk? Risk contingency plan if the risk were to occur what do we do?

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

Test Strategy - Tools Usage


Tools required in the project and rationale to use that particular tool(s) should be documented. Automation requirements should be identified and analyzed
Which phases need to be automated How much of testing needs to be automated

Based on the automation requirements, tools strategy should be prepared.

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

Test Strategy - Templates


Any specific templates to be used for testing apart from the normal templates based on client or project requirements are identified in the test strategy. In some case client provides the templates and well use that.
 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

Sample Strategy doc

Test Strategy - Acceptance criteria


Acceptance test plan can be prepared either by us or by the customer, as agreed. Customer will be involved in preparation, review and approval of acceptance test cases This plan specifies the criteria for client acceptance of the final tested product including Features/functionalities to be tested and traceability information.

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

Test Strategy - Summary


Test strategy is a high level document which states in detail way testing is to be carried out. It is done in line to meet the test and business objectives Tools ,Methods ,and techniques to be used are stated in the test strategy.
 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

Test Planning
       Testing Life Cycle Overview Test Strategy Test Planning Test Case Designing Test Execution Testing Types Testing Tools Overview

Test Planning - Documenting assumptions


If any assumptions are there, it needs to be documented to avoid any confusions
Testers has understanding of V&V-process model or any SDLC model Testers are aware of test phases of the project A good understanding of the Domain Standard naming conventions are followed in documentation

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

Test Planning -Traceability


Requirements tracing is the process of documenting the links between the user requirements for the system you are building and the work products developed to implement and verify those requirements. Work products include software requirements, design specifications, software code, test plans and other artifacts of the systems development process
 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

Traceability Matrix

Test Planning -Traceability


Traceability is documenting of
the dependencies and logical links between individual requirements and other system elements traceability to greatly facilitate change impact analysis. traceability is that it can help identify the propagation of change that can result when a specific requirement is deleted or modified. specific requirements mapped into tasks in a workbreakdown structure, you know those tasks that will be affected if and when a requirement is changed or deleted.

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

Test Planning - Schedule/ Milestones


Test plan captures the details about different milestones of the project and the schedule details. Define the schedule for each event for offshore and onsite. Events may include familiarization, training, test case preparation, test scripting, test execution etc., Completion criteria for each milestone should be identified and documented

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

Test Planning - Resources


Staffing: Check what types of personnel should be considered for inclusion on the test team Check for the skills of the tester Ensure a good ratio of testers to developers. Steps to form the best team: Identify potential test team members Recruit test team members and develop tentative test assignments Define individual work assignments

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

Test Planning - Test run plan


Cumulative TCs Executed

The execution order of test cases depends on many factors and is shown as below:
Impacted Functionality Unstable functionality Customer Priority Functionality Dependency

Release Plan 4
1400 1200 1000 800 600 400 200 0 7/4
4-Jul Plan Actual 106 297

7/5

7/6
5-Jul 257 483

7/7
6-Jul 575 632

7/8
9-Jul 669 737

7/9

7/10 7/11 7/12 7/13


10-Jul 767 855 11-Jul 877 939 12-Jul 983 1015

7/14
13-Jul 1065 1079

7/15 7/16
16-Jul 1119 1140

7/17
17-Jul 1119 1147

# of Days
 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

TC priority

Execution Order Execution rate Execution Order

Test Execution Run Plan

Complexity TC Interdependency

Test Planning - Change management


Version control - Versioning test cases before and after release to customer Change control - Authority to approve and prioritize changes Configuration auditing - Ensure changes made properly Reporting Addition/Updating of test cases/plans because of bugs which are identified by the customer and that are not available in test plans

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

Test Planning Suspension and Resumption criteria


Deadlines (release/testing deadlines etc.) are met and quality of the product is as per the agreed norms. Test cases completed with certain percentage passed Test budget depleted Coverage of code/functionality/requirements reaches specified point Beta or Alpha testing period ends

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

Test Planning Suspension and Resumption criteria


Determination of Test Completion criteria Most popular method of establishing a test phase completion criteria is by detecting a predefined number of bugs or when a pre-defined time elapses E.g.: Unit testing of a unit is not deemed complete unless at least 5 errors are detected Testing completion criteria could be detection and fixing of 30 bugs, or elapsed time of 30 days, whichever is later.

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

Test Planning Suspension and Resumption criteria


Estimation of number of errors How to determine the number of errors to be detected?
An estimate of total number of errors in the program. An estimate of what percentage of these errors can be feasibly found through testing. Estimates of what fraction of errors originated in a particular phase, during what testing phases these are likely to be detected.
 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

Test Planning Suspension and Resumption criteria


Methods of estimation
Rough estimate of total number of errors can be obtained by previous experience on similar programs. Error-seeding techniques involves intention introduction (seeding) of error in the code. After some period of testing time, the product of the number on non-seeded errors found during testing and the ratio of total seeded errors to the number of seeded errors detected during testing, gives an estimate of the number of non-seeded errors remaining in the program.

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

Test Planning Suspension and Resumption criteria


Estimation of errors in a particular phase - Example
Existing data reveals that in large programs, 40% of errors are coding and logic-design mistakes and the rest are generated in earlier phases. Example to illustrate estimation of total number of errors We have a 10,000 statement program, and the number of errors remaining in the program after code review is 5 per 100 statements. Our objective is to find 98% of coding and logic design errors and 95% design errors. Therefore Total number of estimated errors = 500 Coding and Logic design errors = 200 (40% of 500) Design errors = 300. We need to find at least: 196 Coding and Logic Design errors 285 Design errors.

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

Test Planning Suspension and Resumption criteria


Estimation of error distribution in different phases
Coding & Logic Design errors Unit Test 65% Design Errors

0%

Module Integration Test System Test

30%

60%

3%

35%

Total

98%

95%

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

Test Planning Suspension and Resumption criteria


Establishment of completion criteria Exit Criteria for Unit testing would be detection and correction of 130 errors (65% of estimated 130 coding and design logic errors) Exit Criteria for Module Integration testing would be detection and correction of 240 errors (30% of 200 plus 60% of 300) or elapse of one month; which ever is later. Exit Criteria for System Testing is when 111 errors are detected or when 2 months elapses; which ever is later.
Assumption: We have established one month for Integration Testing and 2 months for System Testing.

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

Test Planning Suspension and Resumption criteria


Resolution in case of over estimation of defects In the previous example, let us say that 240 errors could not be detected even after one month of integration testing is completed. Then the project manager should take the following action
With the help of the appropriate tool, he/she should analyze the test cases and decide whether
There are inadequate Test cases, or Excellent Test cases, but lack of errors to detect
 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

Test Planning Suspension and Resumption criteria


Determination of completion criteria through a graphical representation A graph is plotted using number of errors found per unit time during a testing phase. By examining the shape of the curve, one can determine whether to continue the test phase or end it. Assume a program is being Integration tested, and the number of errors per day is plotted. Lets say on the 7h day the graph looks as follows:
 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

Number of errors

AT the end of 7th day, even if the number of errors detected has crossed the criteria mark, it would be wise to continue Integration Testing. This is because, the error rate is still very high.
No. of days 7

Test Planning Suspension and Resumption criteria


Determination of completion criteria through a graphical representation On the hand if the graph looks as shown, then this shows that error rate has decreased, and that testing can be stopped.
Number of errors

No. of days

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

Test Planning Document templates


Typical Test Plan
Test Plan identifier Introduction Test items Features to be tested Features not to be tested Approach Testing strategy Item pass/fail criteria Suspension criteria and resumption criteria Test deliverable Automation scope Testing tasks Effort estimation Environmental needs Roles and Responsibilities

testplan template

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Designing  Test Execution  Testing Types  Testing Tools Overview

Test Case Design


       Testing Life Cycle Overview Test Strategy Test Planning Test Case Design Test Execution Testing Types Testing Tools Overview

Test Case Design (Contd)


Development Of Test Objectives: At very First stage of testing , you need to set the goals and objectives of what you need to achieve. This Simplifies the limits or borders of testing process. Test Objectives will be contain the list of tasks you are going to do and the achievements. Test Case Writing: A specific set of steps and data along with expected results for a particular test objective. A test case should only test one limited subset of a feature or functionality.

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Design  Test Execution  Testing Types  Testing Tools Overview

Test Case Design (Contd)


A typical test case sheet contains these fields Test Case ID: It is unique number given to test case in order to be identified. Test description: The description if test case you are going to test. Revision history: Each test case has to have its revision history in order to know when and by whom it is created or modified. Function to be tested: The name of function to be tested. Priority/Build/time: The priority of the Test Case, Build version of the test case and the Minimum timeto-execute the test case is also mentioned.

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Design  Test Execution  Testing Types  Testing Tools Overview

Test Case Design (Contd)


Environment: It tells in which environment you are testing. Test Setup: Anything you need to set up outside of your application for example printers, network etc., Test Execution: It is detailed description of every step of execution. Expected Results: The description of what you expect the function to do. Actual Results: Pass /Failed. If pass What actually happens when you run the test If failed - Description of what you've observed.

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Design  Test Execution  Testing Types  Testing Tools Overview

Test Case Design (Contd)


Characteristics of a Good Test: Tests are likely to catch bugs No redundancy Not too simple or too complex. TIP: Test case is going to be complex if you have more than one expected results.

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Design  Test Execution  Testing Types  Testing Tools Overview

Test Case Design (Contd)


Test Data Inputs: Before testing, the tester should plan what kind of data he is giving for test. Give data inputs as functional, boundary, stress, performance, usability values etc.
 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Design  Test Execution  Testing Types  Testing Tools Overview

Test Case Design (Contd)


Test Scripts: A test script is the executable form of a test. It defines the set of actions to carry out in order to conduct a test and it defines the expected outcomes and results that are used to identify any deviance in the actual behavior of the program from the logical behavior in the script (errors during the course of that test). In essence it is a program written for a human computer (tester) to execute.
TestCase Templat

TestCase Status Template

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Design  Test Execution  Testing Types  Testing Tools Overview

Test Execution
       Testing Life Cycle Overview Test Strategy Test Planning Test Case Design Test Execution Testing Types Testing Tools Overview

Test Execution
Test execution is culmination of testing activities which involves executing the planned test cases and conducting of the tests. Test execution phase broadly involves execution and reporting.
 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Design  Test Execution  Testing Types  Testing Tools Overview

Test Execution - Activities


Test execution consists of following activities to be performed
Creation of test setup or Test bed Execution of test cases on the setup Test Methodology used Collection of Metrics Defect Tracking and Reporting
 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Design  Test Execution  Testing Types  Testing Tools Overview

Regression Testing

Test Execution Cycle

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Design  Test Execution  Testing Types  Testing Tools Overview

Test Execution - Testing


Only exhaustive testing can show a program that it is defect free ,but exhaustive testing is impossible Tests should exercise the systems capabilities fully Testing typical situations is more important than boundary value conditions

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Design  Test Execution  Testing Types  Testing Tools Overview

Test Execution -Test Data and Test Cases


Test Cases: Inputs to test the system and predicted output from the system are detailed, further will contain the expected result and test topology Test Data: Inputs which are to exercise the system under test.
 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Design  Test Execution  Testing Types  Testing Tools Overview

Test Execution -Types of Testing and Methodologies


There are three major types of testing and methods used are based on the testing type
Unit testing Integration testing System testing

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Design  Test Execution  Testing Types  Testing Tools Overview

Test Execution - Unit Testing


Unit testing is typically done as white box testing. Unit testing is done at module level. Derivation of Test cases are based on the program structure .Knowledge of the program is used to identify the test cases . Objective is to exercise all parts of the program . Test coverage measurements are done to verify that all program parts are exercised Drivers are used to feed the data to the program to test it, wherever required. Test stubs are used to stub for the functions which are outside the module

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Design  Test Execution  Testing Types  Testing Tools Overview

Test Execution - Integration Testing


Tests partial systems composed of integrated components Integration testing is black box testing Main difficulty in integration testing is localizing errors There two approaches to integration testing Top-down approach Bottom - up approach

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Design  Test Execution  Testing Types  Testing Tools Overview

Integration testing- Top Down approach


Start from the top level module and integrate individual components and stub the rest of the system

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Design  Test Execution  Testing Types  Testing Tools Overview

Integration testing - Bottom Up approach


Integrate individual components from lower level module until whole system is integrated .

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Design  Test Execution  Testing Types  Testing Tools Overview

Test Execution - System Testing


System testing is typically Black Box testing where the system is taken as black box and tested based only on its functionality Test cases are based on the system specification It is critical part of the testing process as the system is verified as a whole
 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Design  Test Execution  Testing Types  Testing Tools Overview

Test Execution - Regression Testing


Regression testing is done after fixing all the defects. It is re-testing after fixing the bugs found during testing and also when there is change in the test environment. The impacted portions of the code after fixing the bug is re-tested. Automation of testing helps in faster regression testing.

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Design  Test Execution  Testing Types  Testing Tools Overview

All the defects/bugs found during testing is logged and they are classified based on the severity and priority. They are assigned to developers for fixing. Each defect found is logged and tracked to closure. Test summary report shall be prepared containing the details like numbers of test cases passed/failed ,number of cycles of testing. Generally test defect logging and tracking tools are used for this purpose.

Test Execution - Defect Tracking and Reporting

Test report template

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Design  Test Execution  Testing Types  Testing Tools Overview

Testing is done in rounds. Software passes though several rounds before it gets released. Testing is stopped when there are many major bugs and critical system errors, when further testing is not feasible Testing is stopped when the system becomes un stable during test. When those critical errors are fixed and system is stable, the next round of testing is started . The test cases are executed all over again in each round.

Test Execution - Start / Stop Criteria

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Design  Test Execution  Testing Types  Testing Tools Overview

Test Execution - Metrics


Metrics are important to collect for any project as it helps in monitoring the health of project /product /application under test. Metrics are collected during testing depending on the objective of testing. There are different kinds of metrics which can be used in testing.
For example LOC (Lines of Code) is a metric for Unit testing. i.e., Code related metrics comes in the category of Unit testing.
 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Design  Test Execution  Testing Types  Testing Tools Overview

Test case related metrics can be used for all kinds of testing types, since for every type of test, we will write test cases. Normally source of errors/defects is requirements, design or Implementation. Testing can not be the source of errors/defects.

Test Execution - Metrics


Typical testing metrics
No of LOC tested per day No of test cases executed per day Number of test cases failed Number of bugs reported Number of rejects of bugs reported Number of persons working on the test plan during the month Total Effort spent on test execution in person hours Number of simulations made during the month Number of out of norms per month Average time taken to find a problem categorized by priority Defect density Review efficiency

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Design  Test Execution  Testing Types  Testing Tools Overview

Test Execution - Metrics


Having a complete Metrics data helps in building the Trustworthy, Capable, Reliable and Predictable Product or Application.
Productivity Increase Field error rate Decrease Customer Overhead reduces Improvement release on release On time delivery Risks in Program identified and proactively intimated Decrease in Cost of Quality Decrease in Re-work Helps in identifying the area which needs focus in the project/organization for improvement Helps in arriving at better estimates for the project

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Design  Test Execution  Testing Types  Testing Tools Overview

What makes a good test engineer?


Test Execution Good test engineer?


A good test engineer has a 'test to break' attitude. An ability to take the point of view of the customer A strong desire for quality, and an attention to detail. Tact and diplomacy for maintaining a cooperative relationship with developers. An ability to communicate with both technical (developers) and nontechnical (customers, management) people. Previous software development experience can be helpful, gives the tester an appreciation from the developers' point of view, and reduce the learning curve in automated test tool programming. Judgment skills are needed to assess high-risk areas of an application on which to focus testing efforts when time is limited. Notice little things that others miss/ignore (See symptom not bug) Good testers use files, Databases and all the other accoutrements of an organized mind Testers are fundamentally honest and incorruptible Last but not the least is Patience

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Design  Test Execution  Testing Types  Testing Tools Overview

Test Execution Testing Rules?


Software testing 10 rules?
Test early and test often. Integrate the application development and testing life cycles. Formalize a testing methodology; you'll test everything the same way and you'll get uniform results. Develop a comprehensive test plan; it forms the basis for the testing methodology. Use both static and dynamic testing. Define your expected results. Understand the business reason behind the application. You'll write a better application and better testing scripts. Use multiple levels and types of testing (regression, systems, integration, stress and load). Review and inspect the work, it will lower costs. Don't let your programmers check their own work; they'll miss their own errors.
Source: SQAtester.com

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Design  Test Execution  Testing Types  Testing Tools Overview

Test Execution - Summary


Test execution consists of preparing of test bed/setup ,execution of planned test cases ,verification of results ,Test tracking and reporting and metrics collection. Apart from that ,it involves applying different test methods like black box to system testing ,white box at unit testing level and top down and bottom up in case of integration testing .

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Design  Test Execution  Testing Types  Testing Tools Overview

Test Execution Summary Contd..


Testing will involve several rounds based on the stability and bugs found on the software and start /stop criteria identified . Regression testing will be carried out when ever the new bugs are found and fixed in the tested software to verify the impacted items. Test execution will involve preparation of test reports ,collection of different metrics and meeting the test objectives .

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Design  Test Execution  Testing Types  Testing Tools Overview

Testing Tools Overview


       Testing Life Cycle Overview Test Strategy Test Planning Test Case Design Test Execution Testing Types Testing Tools Overview

Testing Tools Overview (contd)


Testing tools are used to automate the testing process. These tools will have features like
Record and play back Test Script generation Virtual users And so on

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Design  Test Execution  Testing Types  Testing Tools Overview

Testing Tools Overview (contd)


Different vendors released different testing tools in the market. Some of them are Functional / GUI Testing Tools: Win Runner Rational Robot Visual Test

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Design  Test Execution  Testing Types  Testing Tools Overview

Testing Tools Overview (contd)


Performance Testing: Rational Quantify True time Web Testing: Silk Test Astra Quick Test eTest Suite

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Design  Test Execution  Testing Types  Testing Tools Overview

Testing Tools Overview (contd)


Java Testing Tools:
Jtest JProbe QStudio Optimizeit JStyle

 Testing Life Cycle Overview  Test Strategy  Test Planning  Test Case Design  Test Execution  Testing Types  Testing Tools Overview

Appendix 1: Example for test life cycle


To understand the different stages of testing, consider an example in which you need to test a Webpage, say HRWEB->resume part. To test this Webpage following activities needs to be done
Prepare Test Strategy for testing HRWEB->Resume section Prepare Test Plan for testing HRWEB->Resume section Develop Test Cases for testing HRWEB->Resume section Execute the test cases and report the results, including bug reporting, re-test and tracking.

Appendix 1: Example for test life cycle cont..


Test strategy to test HRWEB->Resume page includes overview of
Document the test requirements User Interface testing Test for associated links Error Handling and Exceptions Functionality testing Boundary conditions Test for Load, Performance, Security issues Deciding of tools and test set-up to test

Test Planning to test HRWEB->Resume includes


Resume page and all associated links will be tested Specify how it will be tested - Test Cases Identify resources for testing and finalize timeline Specify Hardware, Software configuration requirement

Appendix 1: Example for test life cycle cont..


Test Case Development
Several Test Cases will be generated as per the test strategy and plan for User Interface, Functionality, Error Handling of Resume Section in HRWEB Integration and System Test cases will be generation to test for the whole application including Resume Section.

Test report:
Test Cases will be executed and Test Results will be recorded and tracked. After bugs are fixed, the system is tested again to verify the same The whole system is also tested to verify that new bugs have not been introduced.

Appendix 1: Example for test life cycle cont..

Sample User Interface Test Case for Resume Section Perform the necessary steps to view the Resume Section on TEDWEB Verify that the menu TEDWEB Menu displays at the top and the Resume Menu displays at the left of the screen Verify the presence of the following buttons - Add Resume, Modify Resume, Add project, Modify Project etc.. Sample Functionality Test Case for Resume Section Click on Add Resume link Verify a new screen appears to the right that allows the user to add his resume details Put the cursor in Educational Qualifications field and verify it is enabled for user input Put the cursor in Summary field and verify it is enabled for user input.

Appendix 1: Example for test life cycle cont..


Sample Error Handling Test Case for Resume Section Click on Add Resume when the resume already exists. Verify the message displayed is Profile Already Exists Click on Modify Resume when no resume has been added. Verify that a proper error message is displayed that no resume exists.

Appendix 2: Testing Standards


IEEE Standard for Software Test Documentation (ANSI/IEEE Standard 829-1983). This is a summary of the ANSI/IEEE Standard 829-1983. It describes a test plan as: A document describing the scope, approach, resources, and schedule of intended testing activities. It identifies test items, the features to be tested, the testing tasks, who will do each task, and any risks requiring contingency planning.

IEEE testplan template

Appendix 3: Additional readings


Please go through the following documents, which are available in Interops KM Page.
Inter101_002-Intro Software Testing Inter101_003-Test Plan & Test Case Inter101_004-SDLC&TLC Inter101_005-Unit & Integration Testing Inter101_006-Functional & Regression Testing Inter101_007-General Testing Types Inter101_008-Test Report Automation Tools Inter101_009-Review & Walk thru Inter101_010-Reliability terminology

S-ar putea să vă placă și