Software Testing Life Cycle (STLC)
Software testing life cycle identifies what test activities to carry out and when (what is the best time) to accomplish the test activities. Even though testing differs in various organizations, there is a testing life cycle.
1. Test Planning
2. Test development
3. Test execution
4. Result Analysis
5. Bug Tracking
6. Reporting
Software testing has its own life cycle that intersects with every stage of the SDLC. The basic requirements in software testing life cycle is to control/deal with software testing – Manual, Automated and Performance.
1. TEST PLANNING:
Test plan is a systematic approach to testing a system such as a machine or software. The plan typically contains a detailed understanding of what the eventual workflow will be.
Test Plan Document generally includes the following:
i. Introduction
-Objective:
Purpose of the document is specified over here.
-References Document:
The list of all the documents that are .(SRS and Project Plan)
ii. Coverage of Testing
-inscope (Features to be tested):
The list of all the features that are to be tested based on the Implicit and explicit requirements from the customer will be mentioned here.- -outscope (Features not to be tested) The list of all the features that can be skipped from testing phase are mentioned here, Generally Out of scope features such as incomplete modules are listed here. If severity is low and time constraints are high then all the low risk features such as GUI or database style sheets are skipped. Also the features that are to be incorporated in future are kept out of testing temporarily.
iii. Test Strategy
-Levels of testing: It’s a project level term which describes testing procedures in an organization. All the levels such as Unit, module, Integration, system and UAT (User Acceptance test) are mentioned here which is to be performed.
-Types of testing: All the various types of testing such as compatibility, regression and etc are mentioned here with module divisions
-Test design techniques: The List of All the techniques that are followed and maintained in that company will be listed out here in this section. Some of the most used techniques are Boundary Value Analysis (BVA) and Equivalence Class Participation (ECP).
-Configuration Management: All the documents that are generated during the testing process needs to be updated simultaneously to keep the testers and developers aware of the proceedings. Also the naming conventions and declaring new version numbers for the software builds based on the amount of change is done by the SCM (Software Configuration Management team) and the details will be listed here.
-Test Metrics: The list of all the tasks that need to be measured and maintained will be present here. Different metrics for tracing back the exact requirement and test case depends on the availability of metrics at the right time.
-Terminology: Testing specific jargons for the project that will be used internally in the company will be mentioned here in this section.
-Automation Plan: The list of all the features or modules that are planned for automation testing will be mentioned here. The application only undergoes automation testing after being declared STABLE by the manual testing team.
-List of Automated tools: The list of Automated tools, like QTP, Load runner, Win runner; etc which will be used in this project will be mentioned along with license details.
iv. Base Criteria:
-Acceptance Criteria: The standards or metrics that need to be achieved by the testing team before declaring the product fit will be listed here. So before handing over to the customer, the time at which the testing needs to be stopped is mentioned here in this section.
-Suspension Criteria: In high risk projects, or huge projects that consists several modules, It is necessary to minimize the repetitive process to be efficient. The situations when the testing needs to be suspended or temporarily halted will be listed here in this section
v. Test deliverables:
The list of all the documents that are to be prepared during the testing process will be mentioned here in this section. All the copies of verification documents after each level are submitted to the customer along with the user manual and product at the end of the project.
Documents include:
1. Test Plan Document
2. Test Scenarios Document
3. Test case Document
4. RTM
5. Defect Reporting Document
6. Final Summary Report
vi.Test environment:
Environmental components and combinations that will be simulated for testing the product will be made sure that it is very close to the actual environment when the end user works on the product. All the details will be mentioned here in this section that is to be used for testing the application.
vii. Resource planning:
Roles to be performed or in other words who has to do what will be mentioned clearly here.
viii. Scheduling:
The starting dates and ending dates for each and every task and module will be listed here in this section.
ix. Staffing and Training:
How much staff is to be recruited and what kind of training is to be provided to accomplish this project successfully will be described here in a detailed fashion.
x. Risks and Contingencies:
The list of all the potential risks and the corresponding solution plans will be listed here
Risks: Example, Resources may leave the organization, license and update deadlines, customer may change the requirements in terms of testing or maintenance in middle of the project etc
Contingencies: Maintaining bench strength, Rechecking initial stages of the whole process, Importance and priority settings for features to be tested and features to be skipped under time constraints to be listed and shared clearly.
xi. Assumptions:
Some features or testing methods have to be done mandatorily even though, the customer does not mention it in his requirements document. These assumptions are listed out here in this section.
xii. Approval Information
As this document is published and circulated, the relevant and required authorities will approve the plan and will update this section with necessary details like date and department etc.
2. TEST DESIGN STAGE:
This is the most important stage of the testing life cycle, in this phase of the testing; the testers will develop the test scenarios and test cases based against the requirements of the customer.
2.1. Test Scenario
A set of test cases that ensure that the business process flows are tested from end to end. They may be independent tests or a series of tests that follow each other, each dependent on the output of the previous one.
The following is the
2.2. Test Case
Test Development simply involves writing Test cases from test scenarios. Every Company follows a particular format for test cases called test case template
Test cases are broadly divided into two types.
· G.U.I Test Cases.
· Functional test cases.
Functional test cases are further divided into two types.
· Positive Test Cases.
· Negative Test Cases.
Test case template typically consists of
Test Case ID: Test Case number
Test case created on: The date test case created on
Category: category deals with type of the testing done like functional, GUI, negative testing etc
Test Scenario: A set of test cases that ensure that the business process flows are tested from end to end. They may be independent tests or a series of tests that follow each other, each dependent on the output of the previous one.
Test Data: If you are writing test case then you need input data for any kind of test. Tester may provide this input data at the time of executing the test cases. The test data may be any kind of input to application, it may be in any format like xml test data, system test data, SQL test data or stress tested.
Generally testers call it as test bed. In test bed all software and hardware requirements are set using the predefined data values. If you don’t have the systematic approach for building test data while writing and executing test cases then there are chances of missing some important test cases
Execution steps: The steps to be followed to execute the test case
Expected Result: When the test case is executed what has happened dictates the expected result
Actual Result: What is actually expected when the test case is executed
Status: The pass or fail criteria
Comments: Any comments can be added.
2.3. Requirement Traceability Matrix
The Test cases ensure that all the functionalities are being covered in testing but these should be mapped with business requirements to ensure that all the requirements are met. For this we need to prepare RTM, to verify 100% Coverage of Requirements.
Requirement Traceability matrix typically consists of
Requirement ID: Maps to the requirement document
Requirement Description: Brief Description of the requirement
Reference Document: The document from where the requirement is fetched like BRD,SRS etc
Module Name: The module in the project where the requirement is under consideration
Test Case ID: The Test case Id from the test case template
Requirement Changes: If any changes are made in the requirements that has to be notified.
3. TEST EXECUTION PHASE:
In this phase test engineers will execute the test cases and log the defects. During the test execution phase the test engineer will do the following.
a) He will perform the action that is described in the description column.
b) He will observe the actual behavior of the application.
c) He will document the observed value under the actual value column.
4. RESULT ANALYSIS:
After the successful execution of test cases, the tester will compare the expected values with the actual values and declare the result as pass or fail.
5. BUG TRACKING AND REPORTING:
It consists of entire list of defects you have come across while testing your application. We have a defects meeting where QA, Developers and PM attend and discuss the defects. If they think it’s a valid defect then QA lead assigns it to development lead who in turn will assign it to the developer to fix the defect.
5.1 Difference between Bug, Defect and Error
Error: Any mismatches while compiling the code is called Error. A human action which produces an incorrect result. In simple words error in programming or coding is an error
Defect: A flow in a component or system that can cause the component or system to fail to perform its required function. The gaps in functionality or any mismatches in functionality are called a defect.
Bug: The defects accepted by developers are bugs.
Fault: Fault is similar to a defect.
Failure: Deviation of the component or system from its expected delivery, service or result.
Issue: It’s same as defect.
Suggestion: The points not covered in BRD but which can improve quality can be suggested by testers to the developers
5.2. Reporting Of bugs
a) Classical Bug Reporting Process:
Drawbacks: 1.Time consuming
2. Redundancy.
3. No Security.
b) Common Repository Oriented Bug Reporting Process:
Drawbacks: 1.Time consuming.
2. Redundancy.
c) Bug Tracking Tool Oriented Bug Reporting Process:
Very Important stage to update the DPD (Defect profile document) and the let the developers know of the defects.
5.3. DPD (Defect Profile Document)
Defect ID: Serial no. of the defect
Defect Description: A brief description of the defect.
Status: Status of the defect
Status - New/Open/Assign/Deferred/Rejected/Re Opened/Closed.
a) New: When the bug is posted for the first time, it states will be new this means the bug is not yet approved.
b) Open: After a tester has posted a bug, the lead of the tester will approves the bug is genuine and he changes the state as “OPEN”.
c) Assign: Once the lead changes the state as “OPEN” he assigns the bug to the corresponding developer or developer team.
d) Deferred: The bug changed to deferred state means the bug is expected to fix in next release due to various factors.
e) Rejected: If the developer feels the bug is not genuine, he rejects the bug. The state of the bug changed to “REJECTED”.
f) Reopened: If the bug still exists even after the bug is fixed by the developer, the tester changes the statues to “RE-OPENED”.
g) Closed: Once the bug is fixed, it is tested by tester. If the tester feels that the bug no longer exists in the software, he changes the status of the bug to “CLOSED”.
Severity: Impact of Bug on the application. The different Severities are Show stopper, Critical, Major, Minor, Cosmetic. It will be decided by the tester.
a) Show stopper: Stops development and/or testing work.
b) Critical: Crashes, loss of data, severe memory leak.
c) Major: Major loss of function.
d) Minor: Minor loss of function, or other problem where easy work around is present.
e) Cosmetic: Cosmetic problem like misspelled words or misaligned text, Background color of the specific field.
Priority: Priority of the Bug as per the client's requirements.
The different priorities are High, Medium and Low.
a) High: Bug must be fixed immediately; the product cannot ship with this bug.
b) Medium: These are important problems that should be fixed as soon as possible. The problem should be fixed within the time available.
c) Low: It is not important that these bugs be addressed.
Fix these bugs after all other bugs have been fixed
Test case ID: Test case Number that is written in the test case Document.
Assigned To: Person (developer) to whom the bug is assigned to fix.
Assigned By: Testing personnel by whom the bug is found or the Testing person who assigns the bug to developer.
Date of testing: The date on which testing is done.
Remarks: Any remarks can be provided.
Once all the bugs are fixed and complete regression testing is done then the final summary report is made.
5.2. Final summary report
6. TEST CLOSURE:
It is the last phase of STLC process. All test cases execution should be completed and there should not be any high level severity and high level priority bugs. Once the UAT has done with successfully then final test summary report is generated and then sign off the project.
· Client Acceptance
· Replication of product delivery records submission client sign-off.
