Unit - 6
Software Reliability & Quality Management
Software reliability was created to predict the number of defects or faults in the software. Software reliability is the probability that the software will execute properly without any failure for a given period of time. Reliability is concerned with how well the software functions to meet the requirements of the customer. It represents a user oriented view of the software quality.
The two terms related to software reliability are-
● Fault: a defect in the software, e.g. a bug in the code which may cause a failure.
● Failure: a derivation of the programs observed behavior from the required behavior.
Software reliability is hard to achieve, because the complexity of software tends to be high. While the complexity of software is inversely related to software reliability it is directly related to other important factors like software quality, especially functionality, capability.
Hardware versus Software Reliability
In any software, industry, system quality plays an important role. It comprises hardware quality and software quality. We know that hardware quality is constantly high. So if the system quality changes, it is because of the variation in software quality only.
Hardware components basically fail due to wear and tear, where as software components fail due to bugs. To fix the hardware fault, we have to either repair or replace the failed parts. On the other hand to fix software fault one has to track down until the error is encountered or the design is changed to fix the bug. Doing this the hardware reliability would remain the same, but software reliability would either increase or decrease.
|
Fig 1: Change in failure rate of a product
Reliability Metrics
Reliability Metrics are used to qualitatively express the reliability of the software product. Some reliability metrics which can be used to quantify the reliability of the software are as follows:
(i) Rate of Occurrence of failure (ROCOF):It is the number of failure appearing in a unit time internal. The number of unexpected events over a specific time of operation . ROCOF is the frequency of occurrence with which unexpected role is likely to appear. For e.g. A ROCOF of 0.05 means that, five failure are likely to occur in 100 operational time unit step.
(ii) Mean time to failure (MTTF):MTTF is described as the time interval between the two successive failures. To measure MTTF, we can evidence data for n failures. Let the failures appear at time instants t1, t2, t3……… tn.
So, MTTF can be calculated as,
An MTTF of 175 means that 1 failure is expected in each 175 time unit
(iii) Mean Time to Repair (MTTR):
MTTR is the average time it takes to track the errors causing the failure and to fix them.
(iv) Mean Time Between failure (MTBF):
The combination of MTTF and MTTR is MTBF.
MTBF = MTTF + MTTR
MTBF of 300 denotes that the next failure is expected to appear after 300 hours.
(v) Probability of Failure on Demand (POFOD):
POFOD is described as the possibility that the system will fail when a service request is made.
A POFOD of 0.1 means that one out of ten service request may fail.
(vi) Availability (AVAIL):Availability is the probability that the system is available for use at a given time. It also takes into account the repair time and the restart time for the system. An AVAIL of 0.995 means that in every 1000 unit time, the system is feasible to be available for 995 of these.The metrics are used to improve the reliability of the system by identifying the areas of requirements.
Reliability Growth Modeling:
A reliability growth model is a mathematical model of how software reliabity improves as errors are detected and repaired.
Jelinski & Moranda model:
The simplest reliability growth model is a step function model where it is assumed that the reliability increases by a constant increment each time errors is deleted and repaired.
|
Fig 2: Step Function Model of Reliability Growth
Little wood and Verall’s model:
This model allows for negative reliability growth to reflect the fact that when a repair is carried out, it may introduce additional errors. It also models the fact that as errors are repaired, the average improvement to the product reliability per repair decreases.
Key takeaway:
● Software reliability was created to predict the number of defects or faults in the software.
● Reliability is concerned with how well the software functions to meet the requirements of the customer.
● Software reliability is hard to achieve, because the complexity of software tends to be high.
The Successful functioning of the software product is important because critical business activities depend on them. Therefore Software quality becomes one of the most important aspects of the software development process. There are many measures of software quality, correctness, maintainability, integrity, and usability provide useful indicators for the project team.
- Correctness.
● Correctness is the degree to which the software performs its required function.
● The most common measure for correctness is defects per KLOC, where a defect is defined as a verified lack of conformance to requirements.
● When considering the overall quality of a software product, defects are those problems reported by a user of the program after the program has been released for general use.
● For quality assessment purposes, defects are counted over a standard period of time, typically one year.
2. Maintainability:
● Maintainability is the ease with which a program can be
● Corrected if an error is encountered,
● Adapted if its environment changes, or
● Enhanced if the customer desires a change in requirements
● There is no way to measure maintainability directly; therefore, we must use indirect measures.
● A simple time-oriented metric is Mean-Time-To Change (MTTC), the time it takes to
● Analyze the change request,
● Design an appropriate modification,
● Implement the change, test it, and
● Distribute the change to all users.
● On average, programs that are maintainable will have a lower MTTC (for equivalent types of changes) than programs that are not maintainable.
3. Integrity:
● This attribute measures a system's ability to withstand attacks (both accidental and intentional) to its security.
● Attacks can be made on all three components of software: programs, data, and documents.
● To measure integrity, two additional attributes must be defined:
● Threat
● Security.
● Threat is the probability (which can be estimated or derived from empirical evidence) that an attack of a specific type will occur within a given time.
● Security is the probability (which can be estimated or derived from empirical evidence) that the attack of a specific type will be repelled.
● The integrity of a system can then be defined as
Integrity = summation [(1 – threat) * (1 – security)]
Where threat and security are summed over each type of attack.
4. Usability:
If a program is not user-friendly, it is often doomed to failure, even if the functions that it performs are valuable. Usability is an attempt to quantify user-friendliness and can be measured in terms of four characteristics:
● The physical and or intellectual skill required to learn the system,
● The time required to become moderately efficient in the use of the system,
● The net increase in productivity measured when the system is used by someone who is moderately efficient, and
● A subjective assessment of user’s attitudes toward the system.
Key takeaway:
● There are many measures of software quality, correctness, maintainability, integrity, and usability provide useful indicators for the project team.
● Software quality one of the most important aspects of the software development process.
Software Quality Management (SQM) is a management process that aims to develop and manage the quality of the software in such a way so as to ensure the product meets the quality standards expected by the customer
● A Quality system is the responsibility of the organization as a whole. Every organization has a separate quality department to perform several quality system activities
● The quality system activities encompasses the following activities
● Auditing of projects
● Review of the quality system
● Development of standards, procedures and guidelines, etc
● Production of reports for the tap management
A good quality system must be well documented as without a proper documentation, the application of quality controls and procedures become adhoc
● Evolution of Quality Systems: Quality System of organization has undergone four stages of evolution. Firstly the product inspection, then quality control followed by quality assurance and then total quality management
● Quality Control: It’s a series of inspections, reviews and tests used throughout the development cycle to ensure that work product meets the requirements placed up on it. It includes a feedback loop to the process that created the work product. The feedback loop helps tune up the process during the creation of the product. It should be a part of the manufacturing process.
● Quality Assurance: It’s the auditing and reporting functions of the management. If the data provided through quality assurance identify problems then it’s the management’s responsibility to solve the problem and apply the necessary resource to resolve the quality issues. The goal of quality assurance is to provide management to provide and data related to product quality so that they can evaluate whether or not the manufacturing process is maintaining the product quality.
● Total Quality Management (TQM) aims at continuous process improvement. The term related to TQM is Business process Reengineering (BPR) which aims at reengineering the way business is carried out in an organization
● Product Metrics versus Process Metrics: Product metrics helps to measure the characteristics of product being developed, whereas process metrics help to measure how a process is performing. Some examples of product metrics are size, complexity, design features, performance and quality level and some example of process metrics are average number of defects, average defect correction time, productivity, average number of failures detected during testing per LOC etc.
|
Fig 3: Evolution of quality system and corresponding shift in the quality paradigm
Key takeaway:
● Software Quality Management (SQM) is a management process that aims to develop and manage the quality of the software.
● Every organization has a separate quality department to perform several quality system activities.
Another popular set of standards related to software quality is the International Organization for Standardization’s (ISO) 9000. ISO is an international standard organization that sets standards for everything from nuts and bolts to, in the case of ISO 9000, quality management and quality assurance.
You may have heard if ISO 9000 or noticed it in advertisements for a company’s products or services. Often it’s a little logo or note next to the company name. It’s a big deal to become ISO 9000 certified, and a company that has achieved it want to make that fact known to its customers – especially if its competitors aren’t certified.
ISO 9000 is a family of standards on quality management and quality assurance that defines a basis set of good practices that will help a company consistently deliver products (or service) that meet their customer’s quality requirements. It doesn’t matter if the company has run out a garage or is a multi – billion - dollar corporation, is making software, fishing lures, or is delivering pizza. Good management practices apply equally to all of them.
ISO 9000 works well for two reasons:
It targets the development process, not the product. It’s concerned about the way an organization goes about its work, not the results of the work. It doesn’t attempt to define the quality level of the widgets coming off the assembly line or the software on the CD. As you’ve learned, quality is relative and subjective. A company’s goals should be to create the level of quality that its customers want. Having a quality development process will help achieve that.
ISO 9000 dictates only what the process requirements are, not how they are to be achieved. For example, the standard says that a software team should plan and perform product design reviews (see chapters 4 and 6), but it doesn’t say how that requirement should be accomplished. Performing design reviews is a good exercise that a responsible design team should do (which is why it’s in ISO 9000), but exactly how the design review is to be organized and run is up to the individual team creating the product. ISO 9000 tells you what to do but not how to do it.
The sections of the ISO 9000 standard that deal with software are ISO 9001 and ISO 9000–3. ISO 9001 is for businesses that design, develop, produce, install, and service products. ISO 9000–3 is for businesses that develop, supply, install and maintain computer software.
The following list will give you an idea of what type of criteria the standard contains. It will also, hopefully, make you feel a little better, knowing that there’s an international initiative to help companies a better software development process and to help them build better quality software.
Some of the requirements in ISO 9000-3 include:
● Develop detailed quality plans and procedures to control configuration management, product verification an validation (testing), nonconformance (bugs), and corrective action (fixes).
● Prepare and receive approval for a software development plan that includes a definition of the project, a list of the project’s objectives, a project schedule, a project specification, a description of how the project is organized, a discussion of risks and assumption, and strategies for controlling it.
● Communicate the specification in items that makes it easy for the customer to understand and to validate during testing.
● Plan, develop, document, and perform software design review procedures.
● Develop Procedures that control software design changes made over the product’s life cycle.
● Develop and document software test plan.
● Develop methods to test whether the software meets the customer’s requirement.
● Perform software validation and acceptance teats.
● Maintain records of the test results.
● Control how software bugs are investigated and resolved.
● Prove that the product is ready before it’s released.
● Develop procedures to control the software’s release process.
● Identify and define what quality information should be collected.
● Use statistical techniques to analyze the software development process.
● Use statistical techniques to Evaluate product quality.
These requirements should all sound pretty fundamental and common sense to you by now. You may even be wondering how a software company could even create software without having these processes in place. It’s amazing that it’s even possible, but it does explain why much of the software on the market is so full of bugs. Hopefully, over time, competition and customer demand will compel more companies in the software industry to adopt ISO 9000 as the means by which they do business.
Key takeaway :
● ISO 9000 is a family of standards on quality management and quality assurance that defines a basis set of good practices
● ISO 9000 dictates only what the process requirements are, not how they are to be achieved.
● ISO 9000 standard that deal with software are ISO 9001 and ISO 9000–3.
The SEI approach provides a measure of the global effectiveness of a company’s software engineering practices and established five process maturity levels that are defined in the following manner:
Level 1: Initial.
The software process is characterized as ad hoc and occasionally even chaotic. Few processes are defined, and success depends on individual effort.
Level 2: Repeatable.
Basic project management processes are established to track cost, schedule, and functionality. The necessary process discipline is in place to repeat earlier successes on projects with similar applications.
Level 3: Defined.
The software process for both management and engineering activities is documented, standardized, and integrated into an organization wide software process. All projects use a documented and approved version of the organization’s process for developing and supporting software. This level includes are characteristics defined for level 2.
Level 4: Managed.
Detailed measures of the software process and product quality are collected. Both the software process and products are quantitatively understood and controlled using detailed measures. This level includes all characteristics defined for level 3.
Level 5: Optimizing.
Continuous process improvement is enabled by quantitative feedback from the process and from testing innovative ideas and technologies. This level includes all characteristics defined for level 4.
The five levels defined by the SEI were derived as a consequence of evaluating responses to the SEI assessment questionnaire that is based on the CMM. The results of the questionnaire are distilled to a single numerical grade that provides an indication of an organization’s process maturity.
The SEI has associated key process areas (KPAs) with each of the maturity levels. The KPAs describe those software engineering functions (e.g., software project planning, requirements management) that must be present to satisfy good practice at a particular level. Each KPA is described by identifying the following characteristics:
- Goals: The overall objectives that the KPA must achieve.
- Commitments: Requirements (imposed on the organization) that must be met to achieve the goals or provide proof of intent to comply with the goals.
- Abilities: Those things that must be in place (Organizationally and technically) to enable the organization to meet the commitments.
- Activities: The specific tasks required to achieve the KPA function.
- Methods for monitoring implementation: The manner in which the activities are monitored as they are put into place.
- Methods for verifying implementation: The manner in which proper practice for the KPA can be verified.
Eighteen KPAs (each described using these characteristics) are defined across the maturity model and mapped into different levels of process maturity. The following KPAs should be achieved at each process maturity level:
- Process maturity level 2
- Software configuration management
- Software quality assurance
- Software subcontract management
- Software project tracking and oversight
- Software project planning
- Requirement management
ii. Process maturity level 3
- Peer reviews
- Inter – group coordination
- Software product engineering
- Integrated software management
- Training program
- Organization process definition
- Organization process focus
iii. Process maturity level 4
- Software quality management
- Quantitative process management
iv. Process maturity level 5
- Process change management
- Technology change management
- Defect prevention
Each of the KPAs is defined by a set of key practices that contribute to satisfying its goals. The key practices are policies, procedures, and activities that must occur before a key process area has been fully instituted. The SEI defines key indicators as “those key practices or components of key practices that offer the greatest insight into whether the goals of key process area have been achieved”.
Key takeaway:
● The five levels defined by the SEI were derived as a consequence of evaluating responses to the SEI assessment questionnaire that is based on the CMM.
● The SEI has associated key process areas (KPAs) with each of the maturity levels.
● The KPAs describe those software engineering functions that must be present to satisfy good practice at a particular level.
● Six sigma is a highly disciplined process that helps us to fours on developing and delivering near perfect products & service.
● Six sigma’s aim is to eliminate waste and inefficiency.
● Six Sigma is a structured approach for –
● Improving processes
● Make the process faster
● Reducing cost
● Lowering defects
● Increasing profits
● Increasing customer satisfaction
All these can be achieved through the use of two six sigma sub-methodologies - DMAIC & DMADV.
DMAIC- It is used to enhance an existing business process. This methodology has five phases:
● Define
● Measure
● Analyze
● Improve
● Control
DMADV- It is used to create new product design or process design, The DMADV as has five phases-
● Define
● Measure
● Analyze
● Design
● Verify
Both six sigma processes are executed by six sigma Green Belts and six sigma Black Belts, and are overseen by six sigma Master Black Belts.
A six sigma method is one in which 99.99966% of all the opportunities to produce some features of a component are statistically expected to be free of defects
1. Statistical Quality Control: Six Sigma is derived from the Greek Letter σ
(Sigma) from the Greek alphabet, which is used to denote Standard Deviation
in statistics. Standard Deviation is used to measure variance, which is an
essential tool for measuring non-conformance as far as the quality of output is
Concerned.
2. Methodical Approach: The Six Sigma is not a merely quality improvement
strategy in theory, as it features a well defined systematic approach of
application in DMAIC and DMADV which can be used to improve the quality
of production. DMAIC is an acronym for Design-Measure- Analyze-Improve-
Control. The alternative method DMADV stands for Design-Measure-
Analyze-Design-Verify.
3. Fact and Data-Based Approach: The statistical and methodical aspect of
Six Sigma shows the scientific basis of the technique. This accentuates
essential elements of the Six Sigma that is a fact and data-based.
4. Project and Objective-Based Focus: The Six Sigma process is
implemented for an organization’s project tailored to its specification and
requirements. The process is flexed to suits the requirements and conditions
in which the projects are operating to get the best results.
5. Customer Focus: The customer focus is fundamental to the Six Sigma
approach. The quality improvement and control standards are based on
specific customer requirements.
6. Teamwork Approach to Quality Management: The Six Sigma process
requires organizations to get organized when it comes to controlling and
improving quality. Six Sigma involving a lot of training depending on the role
of an individual in the Quality Management team.
Key takeaway :
● Six sigma’s aim is to eliminate waste and inefficiency.
● Six sigma is a highly disciplined process that helps us to fours on developing and delivering near perfect products & service.
● DMAIC is used to enhance an existing business process.
● DMADV is used to create new product design or process design.
The Iterative Waterfall model was very common in earlier days to complete a project. But today, when using it to create a program, developers face different problems. During project growth, the key challenges included managing change requests from clients and the high cost and time needed to implement these changes. To counter these limitations of the Waterfall model, the Agile Software Development model was proposed in the mid-1990s
The Agile model was primarily designed to help a project to adapt to change requests quickly. So, the main aim of the Agile model is to facilitate quick project completion. Agility is necessary to achieve this mission. Agility is accomplished by adapting the approach to the project, eliminating behaviours that may not be important for a particular project. Something that is a waste of time and money is also stopped.
The Agile model actually applies to a community of production processes. Such procedures share certain fundamental features, but there are some subtle variations between them. Below are a few Agile SDLC models –
● Crystal
● Atern
● Feature-driven development
● Scrum
● Extreme programming (XP)
● Lean development
● Unified process
The specifications are decomposed into several small parts in the Agile model that can be built gradually. Iterative production is embraced by the Agile model. Over an iteration, each incremental component is created. Each iteration is intended to be small and simple to handle, and can only be completed in a few weeks. One version is scheduled, produced and deployed to the clients at a time. They do not make long-term plans.
The fusion of iterative and incremental process models is the Agile model. In agile SDLC models, the steps involved are:
● Requirement gathering
● Requirement Analysis
● Design
● Coding
● Unit testing
● Acceptance testing
The time for an iteration to be completed is known as a Time Box. The time-box refers to the maximum amount of time available for clients to deliver an iteration. So, it does not change the end date for an iteration. But, if required to produce it on time, the development team may decide to decrease the supplied functionality during a Time-box.
Extreme programming :
On these tasks and coding, Intense Programming builds. With many tight feedback loops through efficient implementation, testing and continuous refactoring, it is the comprehensive (not the only) design operation.
Extreme Programming is based on the values below:
● Communication
● Simplicity
● Feedback
● Courage
● Respect
XP is a software development that is lightweight, effective, low-risk, scalable, predictable, science and fun.
In the face of unclear and evolving requirements, eXtreme Programming (XP) was conceived and developed to meet the unique needs of software development by small teams.
One of the methodologies of Agile software development is Extreme Programming. It offers beliefs and ideals to guide the actions of the team. It is predicted that the team will self-organize. Extreme programming offers particular core activities in which
● Each activity is easy and complete in itself.
● More complex and changing behaviour is created by a combination of techniques.
Extreme Programming involves −
● Prior to programming, writing unit tests and keeping all of the tests running at all times. Unit testing is automated and faults are eliminated early, thus lowering costs.
● Just enough to code the features at hand and redesign when necessary, beginning with a basic design.
● Pair programming (called pair programming), with two programmers on a single computer, using the keyboard in turns. When one of them is on the keyboard, the other continuously reviews and gives suggestions.
● Several times a day, implementing and checking the entire system.
● Rapidly bringing a minimal working system into development and updating it whenever appropriate.
● Holding the client active and receiving constant input all the time.
As the programme progresses with the evolving demands, iterating enables the accommodating changes.
Key takeaway:
● The basic concept of the Agile model is that the consumer receives an increase after each time box.
● The Agile model actually applies to a community of production processes.
● Agility is accomplished by adapting the approach to the project, eliminating behaviours that may not be important for a particular project.
● XP is a software development that is lightweight, effective, low-risk, scalable, predictable, science and fun.
Agile project management is an iterative project management approach that focuses on breaking down broad projects into more manageable tasks that are completed during the project life cycle in short iterations. Teams that adopt the Agile approach are able to complete work more quickly, adapt to changing project demands, and optimise their workflow.
As the name suggests, Agile enables teams to be better equipped to change direction and concentration quickly. In particular, software companies and marketing agencies are aware of the tendency for week-to-week changes from project stakeholders.
The Agile methodology allows teams to re-evaluate the work they are doing and adjust in certain increments to ensure that the focus also changes for the team as the work and customer landscape changes.
Key Components of Agile Project Management
- User stories : Simply put, a user storey is a high-level definition of a request for work. It contains just enough data so that the team can produce a reasonable estimate of the effort needed to fulfil the request. From the perspective of the user, this short, simple description is written and focuses on outlining what your customer wants (their goals) and why.
2. Sprints: Sprints are a short iteration in which teams work on tasks determined in the sprint planning meeting, usually between one to three weeks to complete. As you move forward, the concept is to repeat these sprints continuously until the feature is ready for your product. You check the product once the sprint is over, see what is and isn't working, make adjustments, and start another sprint to enhance the product or service.
- Stand-up meeting
Daily stand-up meetings, also known as "daily Scrum meetings" (under 10 minutes), are a great way to ensure that everyone is on track and informed. Such daily experiences are referred to as "stand up" because the participants are required to remain stationary, helping to keep the meetings short and to the point.
2. Agile board
An Agile board allows your team to monitor your project's progress. This can be a sticky notes whiteboard, a simple Kanban board, or a feature within your software for project management.
3. Backlog
They become outstanding stories in the backlog as project requests are added through your intake system. Your team will estimate storey points for each assignment during Agile scheduling sessions. Stories in the backlog are moved into the sprint during sprint planning, to be completed during the iteration.
Key takeaway:
● Agile project management is an iterative project management approach that focuses on breaking down broad projects into more manageable tasks
● Teams that adopt the Agile approach are able to complete work more quickly, adapt to changing project demands, and optimise their workflow.
References :
- Software Engineering by Jan Sommerville (9th Edition) Pearson
2. Software Engineering Fundamentals –Behforooz & Hudson (Oxford: Indian Edition 1st)
3. Software Engineering: A precise Approach – Pankaj Jalote (Wiley India)
