The Real World of Software Testing
August 26, 2003
Reviews, Inspections, and Walkthroughs
In a review , a work product is examined for defects by individuals other than the person who produced it. A Work Product is any important deliverable created during the requirements, design, coding, or testing phase of software development.
Research shows that reviews are one of the best ways to ensure quality requirements, giving you as high as a 10 to 1 return on investment. Reviews help you to discover defects and to ensure product compliance to specifications, standards or regulations
Software Inspections are a disciplined engineering practice for detecting and correcting defects in software artifacts, and preventing their leakage into field operations.
Software Inspections are a reasoning activity performed by practitioners playing the defined roles of Moderator, Recorder, Reviewer, Reader, and Producer.
Moderator: Responsible for ensuring that the inspection procedures are performed through out the entire inspection process. The responsibilities include
Recorder: The Recorder will document all defects that arise from the inspection meeting. This documentation will include where the defects was found. Additionally, every defect is assigned a defect category and type.
Reviewer: All of the Inspection Team individuals are also considered to play the Reviewer role, independent of other roles assigned. The Inspector role is responsible for analyzing and detecting defects within the work product.
Reader: The reader is responsible for leading the Inspection Team through the inspection meeting by reading aloud small logical units, paraphrasing where appropriate
Producer: The person who originally constructed the work product. The individual that assumes the role of Producer will be ultimately responsible for updating the work product after the inspection.
In a Walkthrough, the producer describes the product and asks for comments from the participants. These gatherings generally serve to inform participants about the product rather than correct it.
The Software Inspection Process
Great place for Software Inspection Process. This site will give you the detailed description of all stages in Software Inspection Process. It also lists different reports produced after inspection.
August 22, 2003
Defect Management Process
Keeping in mind the philosophies and goals developed in QAI research report number 8, Mosaic Inc. developed a multi-step approach to defect management. The major steps involved in the process are:
Defect Prevention
Implementation of techniques, methodology, and standard processes to reduce the risk of defects
Identify the critical risks facing the project or system. These are the types of defects that could jeopardize the successful construction, delivery and/or operation of the system.
For each critical risk, make an assessment of the financial impact if the risk becomes a problem.
Once the most important risks are identified try to eliminate each risk. For risks that cannot be eliminated, reduce the probability that the risk will become a problem and the financial impact should that happen.
Deliverable Baseline
A deliverable (e.g. work product) is baselined when it reaches a predefined milestone in its development.
Errors caught before a deliverable is baselined would not be considered defects.
Deliverable baselining involves the following activities:
Select those deliverables that will be baselined and the point within the development process where the deliverable will be baselined.
Set the requirements for each deliverable and the criteria that must be met before the deliverable can be baselined.
Defect Discovery
Discover the Defects before they become problem
Techniques to find defects can be divided into three categories
Static Techniques – Code review
Dynamic Techniques - Executing Test Cases
Operational Techniques – Defects found by users, customers, or control personnel
Defect Resolution
Developers determine the importance of fixing a particular defect. It is a three level method
Critical
Major
Minor
Developers schedule when to fix a defect. Then developers should fix defects in order of importance
Developers notify all relevant parties how and when the defect was repaired along with other pertinent information such as:
The nature of the fix,
When the fix will be released, and
How the fix will be released.
Process Improvement
Management Reporting (Parallel activity for the above 5 steps)
It is important that the defect information, which is a natural by-product of the defect management process, be analyzed and communicated to both project management and senior management. This could take the form of defect rates, defect trends, types of defects, failure costs, etc. From a tactical perspective, Defect Arrival Rate (rate at which new defects are being discovered) is a very useful metric that provides insight into a project's likelihood of making its target date objectives. Defect Removal Efficiency is also considered to be one of the most useful metrics; however it can not be calculated until the system is installed. Defect Removal Efficiency is the ratio of defects found prior to product operation divided by the total number of defects found in the application.
August 20, 2003
What is Quality?
The definition of the term quality is an issue. Based interesting discussion of the meaning of Quality, a surprising number of people still think software quality is simply the absence of errors. Dictionary definitions are too vague to be of much help. The only relevant definition offered by the Oxford English Dictionary (Oxford, 1993), for instance, is peculiar excellence or superiority. Noteworthy here is that quality cannot be discussed for something in isolation: comparison is intrinsic.
Many software engineering references define software quality as correct implementation of the specification. Such a definition can be used during product development, but it is inadequate for facilitating comparisons between products. Standards organizations have tended to refer to meeting needs or expectations, e.g. the ISO defines quality as the totality of features and characteristics of a product or service that bears on its ability to satisfy stated or implied needs.
IEEE defines quality as (1) The degree to which a system, component, or process meets specified requirements. (2) The degree to which a system, component, or process meets customer or user needs or expectations. An older IEEE defines Software quality is the degree to which software possesses a desired combination of attributes.
Quality has been variously defined as:
In short these six definitions show different aspects of quality. All can be applied to software development. We often find our products marketed for their excellence. We want to delight our customers with our products to build a long term business relationship. Many countries trade laws oblige us to sell the product only when fit for the purpose to which our customer tells us they will put it. When purchasing managers look at our software, they may judge comparable products on value knowing that this may stop them buying the excellent product. In managing the software development, efficiency and effective development processes together help avoid losses through rework and reducing later support and maintenance budgets. In testing, we work to see that the product conforms to specification.
Thanks to Carol Long
