Software Quality Assurance

1.1 Software Quality factors

Operational characteristics:

- correctness - does it do what I want?

- reliability - does it do it accurately?

- efficiency - will it run efficiently on my hardware?

- integrity - is it secure?

- usability - is it designed for the user?

Product revision:

- maintainability - can I fix it?

- flexibility - can I change it?

- testability - can I test it?

Product transition:

- portability - will I be able to use it on another machine?

- reusability - will I be able to reuse some of the software?

- interoperability - will I be able to interface it with another system?

Software Quality Assurance

1. Software Quality

1.1. Definition

Software quality is called the conformance to explicitly stated functional

and performance requirements, documented development standards,

and implicit characteristics.

Important points:

- software requirements are the foundation from which quality is measured ;

- specified standards define development criteria that guide the manner

   in which the software is engineered ;

- if the software meets only the explicit requirements, and does not meet

   the implicit requirements, the software quality is suspect

Software Engineering

The Software Engineering Institute (SEI) is a federally funded research and development center, operated by CarnegieMellon University under contract with the United States Department of Defense.
 The SEI Education Program is developing a wide range of materials to support software engineering education. A curriculum module identifies and outlines the content of a specific topic area, and is intended to be used by an instructor in designing a course. A support materials package includes materials helpful in teaching a course. Other materials under development include textbooks and educational software tools.
 SEI educational materials are being made available to educators throughout the academic, industrial, and government communities. The use of these materials in a course does not in any way constitute an endorsement of the course by the SEI, by Carnegie Mellon University, or by the United States government.
 SEI curriculum modules may be copied or incorporated into other materials, but not for profit, provided that appropriate credit is given to the SEI and to the original author of the materials.
 Requests for additional information should be addressed to the Director of Education, Software Engineering Institute, Carnegie Mellon University, Pittsburgh, Pennsylvania 15213.
 Comments on SEI materials are solicited, and may be sent to the Director of Education, or to the module author.

 PPT] Software Metrics download

The People Side of the Metrics Equation

No discussion on selecting, designing and implementing software metrics would be complete without a look at how measurements affect people and people affect measurements. Whether a metric is ultimately useful to an organization depends upon the attitudes of the people involved in collecting the data, calculating, reporting, and using the metric.  The simple act of measuring will affect the behavior of the individuals being measured. When something is being measured, it is automatically assumed to have importance.  People want to look good; therefore, they want the measures to look good. When creating a metric, always decide what behaviors you want to encourage. Then take a long look at what other behaviors might result from the use or misuse of the metric. The best way I have found to avoid human factors problems in working with metrics is to follow some basic rules:

Don't measure individuals: The state-of-the-art in software metrics is just not up to this yet. Individual productivity measures are the classic example of this mistake. Remember that we often give our best people the hardest work and then expect them to mentor others in the group. If we measure productivity in lines of code per hour, these people may concentrate on their own work to the detriment of the team and the project. Even worse, they may come up with unique ways of programming the same function in many extra lines of code. Focus on processes and products, not people.

Never use metrics as a "stick": The first time we use a metric against an individual or a group is the last time we get valid data.

Don't ignore the data: A sure way to kill a metrics program is to ignore the data when making decisions. "Support your people when their reports are backed by data useful to the organization" [Grady-92].  If the goals we establish and communicate don't agree with our actions, then the people in our organization will perform based on our behavior, not our goals.

Never use only one metric: Software is complex and multifaceted. A metrics program must reflect that complexity.  A balance must be maintained between cost, quality and schedule attributes to meet all of the customer's needs. Focusing on any one single metric can cause the attribute being measured to improve at the expense of other attributes.

Target Goals

Basili and Rombach [Basili-88] define a Goal/Question/Metric paradigm that provides an excellent mechanism for defining a goal-based measurement program.  Figure 3 illustrates the Goal/Question/Metric paradigm.

The second step in setting up a metrics program is to select one or more measurable goals. The goals we select to use in the Goal/Question/Metric will vary depending on the level we are considering for our metrics. At the organizational

level, we typically examine high-level strategic goals like being the low cost provider, maintaining a high level of

customer satisfaction, or meeting projected revenue or profit margin target.  At the project level, we typically look at goals that emphasize project management and control issues or project level requirements and objectives.  These goals typically reflect the project success factors like on time delivery, finishing the project within budget or delivering software with the required level of quality or performance. At the specific task level, we consider goals that emphasize task success factors.  Many times these are expressed in terms of the entry and exit criteria for the task.

Software metrics programs must be designed to provide the specific information necessary to manage software projects and improve software engineering processes and services. Organizational, project, and task goals are determined in advance and then metrics are selected based on those goals. The metrics are used to determine our effectiveness in meeting these goals.

When talking to our customers, we may find many of their individual needs are related to the same goal

or problem but expressed from their perspective or in the terminology of their specialty. Many times, what we hear is their frustrations.

For example, the Project Manager may need to improve the way project schedules are estimated. The Functional Manager is worried about late deliveries.  The practitioners complain about overtime and not having enough time to do things correctly.  The Test Manager states that by the time the test group gets the software it’s too late to test it completely before shipment.

When selecting metrics, we need to listen to these customers and, where possible, consolidate their various goals or problems into statements that will help define the metrics that are needed by our organization or team.

In our example, all these individuals are asking for an improved and realistic schedule estimation process