Unit 5
Software Quality and Testing
Quality in software is an abstract term. It can be hard to identify its existence, but its absence can be easy to immediately see. Therefore, in the quest to enhance software quality, we must first consider the concept of software quality.
“Software quality tests, in the sense of software engineering, how well the software is built (design quality) and how well the software conforms to that design' (quality of conformance). The 'fitness for purpose' of a piece of software is also represented.”
The quality of software is described as a field of study and practice which describes the desirable characteristics of software products.
Software quality has two primary approaches: defect control and quality attributes:
- Software quality defect management approach
Any failure to address end-user specifications may be called a programme error. Common defects include specifications and errors missing or misunderstood in design, functional logic, data relationships, process timing, validity testing, and errors in coding.
The approach to handling software defects is focused on counting and managing defects. Defects are typically classified by severity, and preparing uses the numbers in each group.
In order to quantify and enhance production process capability, more advanced software development organizations use methods such as defect leakage matrices (to count the number of defects that pass-through development phases prior to detection) and control charts.
b. Software quality attribute approach
The fixed quality models, such as ISO/IEC 25010:2011, best demonstrate this approach to software quality. A hierarchy of eight quality features, each composed of sub-characteristics, is defined in this standard:
● Functional suitability
● Reliability
● Operability
● Performance efficiency
● Security
● Compatibility
● Maintainability
● Transferability
Fig 1: Software quality model
You might think that the most characteristic left, Functional Suitability, is equivalent to a functional requirement at first glance, but it's not. Functional completeness, functional correctness and functional suitability are sub-characteristics that refer to functions that have been introduced and are characteristics of such functions.
Key takeaways
- The quality of software is described as a field of study and practise which describes the desirable characteristics of software products.
- Software quality has two primary approaches.
The term "metrics" refers to measurement criteria. Software Quality Metrics means the calculation of attributes related to the quality of software along with its production process.
The phrase "software quality metrics" explains the picture by documenting the number of bugs or security loopholes present in the software to calculate the software properties. Quality assessment, however, is not limited to the counting of faults or vulnerabilities, but also includes other types of characteristics, such as repair, reliability, integrity, accessibility, customer satisfaction, etc.
Basically, it is a subset of software metrics that focuses primarily on software product, process and project quality properties:
❏ Product Metrics: It involves scale, shape, complexity, efficiency and other parameters that are related to the quality of the product.
❏ Process Metrics: These features may be used to boost the software's production and maintenance activities.
❏ Project Metrics: The metrics define the functionality and execution of the project. It can include the number of teams, developers involved, project costs and duration, etc.
Fig 2: Types of metrics
Features of good software quality metrics:
● The same attribute or an attribute of greater significance should be specific for calculating.
● A broad range of scenarios are comprehensive.
● Attributes that have already been measured by a particular metric should not be considered.
● Reliable to function in all situations in a similar way.
● It should be quick and easy to comprehend and work.
Key takeaways
- The term "metrics" refers to measurement criteria.
- Software Quality Metrics means the calculation of attributes related to the quality of software along with its production process.
The dilemma of software quality is a situation in which there is doubt as to what we should prioritize: high quality work or fast-paced work. There will be deadlines to meet in software development several times, in such situations software quality dilemmas are bound to occur.
Without proper optimization or feedback, a developer will have to choose between writing and optimized and well commented code or just get the job done. In the same way, many businesses have to determine or skip a product for good software quality between periodic evaluations and expert opinions to meet budgets and deadlines.
● “Good enough” software
Good enough software provides high-quality features and features that are needed by end-users, but at the same time provides other more obscure or specialized features and functions containing known bugs.
- Known bugs
- Time to market
- Different domain
- Legal penalties
● Cost of quality
- Prevention cost include:
➢ Quality planning
➢ Training
➢ Test planning cost
➢ Technical activity
- Failure cost
➢ Internal failure cost
➢ External failure cost
- Appraisal cost
➢ Technical review
➢ Testing and debugging
● Risk
- Technology of low quality raises the risk of developing uses
- Sometimes very serious risks
● Negligence and liability
- Customer / user VS developer
● Quality and security
- Secure data
● Impact of management actions
- Cost and schedule estimate
For your software company, achieving quality will ensure maximum benefit. But achieving consistency is the biggest challenge and here are some of the ways.
● Defining features that decide the quality of a product
● Decide how to quantify each of the quality characteristics
● Setting criteria for each characteristic of quality
● Do quality management with regard to the requirements
● Find out the factors that impede quality
● Make changes that are required
Testing is the method of running a programme with the intention of discovering mistakes. It needs to be error-free to make our applications work well. It will delete all the errors from the programme if testing is performed successfully.
The method of identifying the accuracy and consistency of the software product and service under test is software testing. It was obviously born to verify whether the commodity meets the customer's specific prerequisites, needs, and expectations. At the end of the day, testing executes a system or programme to point out bugs, errors or faults for a particular end goal.
Software testing is a way of verifying whether the actual software product meets the expected specifications and ensuring that the software product is free of defects. It requires the execution of software/system components to test one or more properties of interest using manual or automated methods. In comparison to actual specifications, the aim of software testing is to find mistakes, gaps or missing requirements.
Some like to claim software testing as testing with a White Box and a Black Box. Software Testing, in simple words, means Program Verification Under Evaluation (AUT).
Benefits of software testing
There are some pros of using software testing:
● Cost - effective: It is one of the main benefits of software checking. Testing every IT project on time allows you to save the money for the long term. It costs less to correct if the bugs were caught in the earlier stage of software testing.
● Security: The most vulnerable and responsive advantage of software testing is that people search for trustworthy products. It helps to eliminate threats and concerns sooner.
● Product - quality: It is a necessary prerequisite for any software product. Testing ensures that consumers receive a reliable product.
● Customer - satisfaction: The main goal of every product is to provide its consumers with satisfaction. The best user experience is assured by UI/UX Checking.
Key takeaways
- The method of identifying the accuracy and consistency of the software product and service under test is software testing.
- Some like to claim software testing as testing with a White Box and a Black Box.
- Software Testing, in simple words, means Program Verification Under Evaluation.
Software testing is a method of running a programme in order to detect an error. It should be error free in order to make our programme work well. It will delete all the errors from the programme if testing is performed successfully.
In software testing, there are seven principles:
- Testing shows presence of defects
The purpose of evaluating software is to make the software fail. Checking with software decreases the presence of defects. Testing software speaks about the existence of defects and doesn't speak about the lack of defects.
Computer testing can verify the presence of defects, but it cannot guarantee the software is free of defects. Even several checks will never guarantee the software is 100 percent free of bugs. Testing will decrease the number of defects, but it does not eradicate all defects.
2. Exhaustive testing is not possible
It is the method of checking all potential inputs (valid or invalid) for the functionality of a programme and pre-conditions are known as exhaustive testing. Comprehensive testing is difficult, because the programme can never be tested in any test case.
Only certain test cases can be checked and the programme is presumed to be accurate and the correct output will be generated in any test case. If all test cases are evaluated by the programme, then it will require more cost, effort, etc. and that is impractical.
3. Early testing
Early test operations must be initiated in order to detect the defect in the software. It would be much less costly to find a defect in the early phases of SDLC. Software testing will start at the initial stage for improved software results, i.e. testing will be done at the stage of the requirement review.
4. Defect clustering
In a project, most of the defects may be found in a limited number of modules. The Pareto Principle for software testing states that 20 percent of modules account for 80 percent of software defects.
5. Pesticide paradox
Fresh vulnerabilities will not be discovered by running the same test cases again and again. So, the test cases need to be checked and test cases inserted or changed to identify new bugs.
6. Testing is context dependent
The approach to testing depends on the context of the developed software. Similar types of software need to perform different types of testing. The testing of an e-commerce platform, for example, varies from the testing of an Android application.
7. Absence of errors fallacy
If a designed programme is 99% bug-free, but does not comply with the specifications of the customer, then it is unusable. Not only is it important for software to be 99% bug-free, but it is also mandatory to satisfy all customer requirements.
Key takeaways
- Testing will decrease the number of defects, but it does not eradicate all defects.
- Even several checks will never guarantee the software is 100 percent free of bugs.
A Test Plan is a comprehensive document outlining the test method, targets, timetable, estimation, deliverables, and resources needed for a software product to be evaluated. Test Plan helps us assess the effort required to verify the consistency of the application under test. As a specified process that is minutely supervised and managed by the test manager, the test plan serves as a model for conducting software testing activities.
How to write test plan
To formulate an effective test plan, you should follow these steps:
Fig 3: steps of test plan
Step 1: Analyze the test product
It's not possible to test a product without some details about it. Therefore, the first step in designing a test plan is to assess the product, its characteristics, and features. You need to provide a proper understanding of the demands and preferences of users. To get a deep understanding of the project, here are the three easy steps.
Step2: Develop Test Strategy
You can devise a test plan after you have evaluated the product, which will help you determine the scope of testing. Several research methods can be part of the test plan. Bearing in mind the user requirements, you can decide on which form to use.
For example, you can include 'Load Testing' in your test plan if you are constructing a website that has thousands of online users.
Four further steps, as shown below, also involve identifying a test strategy.
Fig 4: steps for develop test strategy
Step 3: Define Objectives
The goal of every software project is to deliver a bug-free software product of high quality to the market on time. So, you need to specifically identify the research scope and its limits as part of a test plan.
To establish test goals, you should only follow these two basic steps:
● List all the features of software that may need testing
● Defines the target of the test, based on previously mentioned features
Step 4: Resource Planning
Basically, the resource plan is a comprehensive description of all sorts of resources required to accomplish the mission of the project. The resources here could be people, equipment and supplies needed to complete a project. It is a vital phase that allows you to decide the amount of resources to be used for the project.
Step 5: Develop a Schedule
Your next move is to build a plan for testing with the knowledge of the testing approach and scope in hand. Creating a plan allows you to monitor the progress of the process of testing.
You should consider factors such as: when drawing up a schedule:
● Project estimate
● Project risk estimate
● Resource estimate
● Employee roles & responsibilities
● Test activity deadlines
Step 6: Determine Test Deliverables
Your next move is to build a plan for testing with the knowledge of the testing approach and scope in hand. Creating a plan allows you to monitor the progress of the process of testing.
You should consider factors such as, when drawing up a schedule:
Fig 5: different phase of test deliverable
Key takeaways
- A test plan in software testing is a document detailing a testing project's what, where, how, who, and more.
- The first phase of the software testing process is preparation.
- The preparation for the whole test process is outlined in a test plan paper.
- Test Plan helps us assess the effort required to verify the consistency of the application under test.
A test case is a document that has a set of conditions or behaviour carried out on the software application to check the feature's intended functionality.
Test cases are the second most comprehensive way to record research work, after test scripts. Without describing the specific steps to be taken or the data to be used, they identify a particular concept to be tested.
For example, you record something like 'Test if discounts can be applied to the real price' in a test case. This does not address how the coupons should be applied or whether there are other ways to apply. It also doesn't say whether a connection is used by the tester to apply a discount, enter a code, or have it applied by a customer service. They give the tester freedom to determine how they want the test to be carried out.
Benefits of writing test case:
The main aim of a test case is to ensure that various features function as intended within an application. It allows the tester to verify whether the programme is free of bugs and whether it functions according to the end user's expectations.
Further advantages of test cases include:
● Test cases ensure good test coverage
● Help improve the quality of software
● Requires the tester to think carefully from as many angles as possible and approach the tests
● Test cases are reusable for the future, and the test can be referenced and carried out by anyone.
Test case format
An ID, definition, bunch of inputs, few actionable steps, as well as anticipated and actual results are the primary ingredients of a test case. Let's learn what every single one of them is:
● Test Case Name: A test case needs to have a self-explanatory name or title.
● Test Case Description: The definition should inform the tester what they would briefly assess.
● Pre-Conditions: Any assumptions that relate to the test should be mentioned here and any preconditions that must be met before the test is conducted.
● Test Case Steps: The required details and information on how to perform the test should be included in the test steps. The measures, without leaving out important details, should be transparent and brief.
● Test Data: Choosing a data set that gives ample coverage is essential. Select a data set that not only defines positive possibilities, but also negative ones.
● Expected Result: As a consequence of the test steps, the anticipated findings inform the tester what they might experience.
● Actual Result: They define how the programme actually behaved during the execution of test cases.
● Comments: Any other useful data that the tester needs to define can be used here, such as screenshots.
Test Case Design Techniques
An efficient test case design technique is necessary to improve the quality of the software testing process. It helps to improve the overall quality and effectiveness of the released software. The test case design techniques are broadly classified into three major categories:
❖ Specification-Based (Black Box Techniques)
❖ Structure-Based (White Box Techniques)
❖ Experienced-Based Techniques
Key takeaways
- Test cases are the second most comprehensive way to record research work, after test scripts.
- The design of the test case involves preconditions, case name, input conditions, and anticipated outcome.
- A test case is a first stage action and extracted from test scenarios.
The most popular types of testing applications
- Unit testing
Testing each part or module of your software project is known as unit testing. Knowledge of programming is important to conduct this kind of research. So, this kind of research is only performed by programmers, not testers.
As you can test each and every unit of code in your project, you must do a great deal of unit testing.
2. Integration testing
The aim is to take components tested by the unit and construct a software structure that has been determined by design. Integration testing is testing in which a group of components is combined to generate performance.
3. Regression testing
If you need to make adjustments to any part, module, or feature, you need to see if, after such modifications, the whole system works correctly. Testing of the entire system is known as regression testing after such modifications.
4. Alpha testing
This is a form of validation testing. It is a type of acceptance testing that is conducted before the product is released to clients. It is usually done by QA citizens.
5. Beta testing
After alpha testing, beta testing takes place. Before the launch of the product, beta testing is completed. It is carried out by a small number of actual customers or users in a specific user environment, in order to make sure that the programme is fully error-free and operates smoothly.
6. System testing
This software is tested in such a way that the various operating systems operate fine. Under the black box testing process, it is sealed. In this, without concentrating on internal working, we simply concentrate on the necessary input and output.
7. Performance testing
Quality tests are performed to verify whether or not the performance of the programme is good. Performance testing tools exist that analyse the performance of your app and show you the performance problems. You would be able to improve the efficiency of your software application by correcting those problems.
8. Usability testing
Testing an app's user-friendliness is referred to as usability testing. It involves checking how well the app is functional or user-friendly. It checks whether any user can use your software easily without getting stuck. One of the easiest ways to test your software's usability is to invite a few people to use your software. Without taking any assistance from you, see if they can do such items in your app.
9. Compatibility testing
It is designed to assess software's run-time output in the sense of an integrated framework. It is used to measure the program's speed and performance. It is called load checking, too. In it, we check what the system's output is in the given load.
Verification and validation are the processes of investigating whether a software system meets requirements and specifications and fulfils the function necessary.
Barry Boehm defines the verification and validation -
Verification: Are we building the product, right?
Validation: Are we building the right product?
Fig 6: validation and verification
Verification:
Verification is the method of testing without any bugs that a programme achieves its target. It is the method of ensuring whether or not the product that is produced is right. This tests whether the established product meets the specifications we have. Verification is Static Testing.
Verification tasks involved:
- Inspections
- Reviews
- Walkthroughs
- Desk-checking
Validation: Validation is the process of testing whether the software product is up to the mark or, in other words, the specifications of the product are high. It is the method of testing product validation, i.e. it checks that the correct product is what we are producing. Validation of the real and expected product is necessary. Validation is the
Dynamic Testing.
Validation tasks involved:
- Black box testing
- White box testing
- Unit testing
- Integration testing
Key takeaways
- Validation is preceded by verification.
- Verification is the method of testing without any bugs that a programme achieves its target
- Validation is the process of testing whether the software product is up to the mark
Software Testing is a method of assessing a software application's performance to detect any software bugs. It checks whether the developed software meets the stated specifications and, in order to produce a better product, identifies any defects in the software.
In order to find any holes, errors or incomplete specifications contrary to the real requirements, this is effectively executing a method. It is also defined as a software product verification and validation process.
To ensure that every component, as well as the entire system, works without breaking down, an effective software testing or QA strategy involves testing at all technology stack levels. Some of the Techniques for Software Testing include:
Leave time for fixing: When problems are identified, it is necessary to fix the time for the developers to solve the problems. Often, the business also needs time to retest the fixes.
Discourage passing the buck: You need to build a community that will allow them to jump on the phone or have desk-side talk to get to the bottom of stuff if you want to eliminate back and forth interactions between developers and testers. Everything about teamwork is checking and repairing.
Manual testing has to be exploratory: If any problem can be written down or scripted in specific terms, it could be automated and it belongs to the automated test suite. The software's real-world use will not be programmed, and without a guide, the testers need to break stuff.
Encourage clarity: You need to create a bug report that does not create any uncertainty but offers clarification. It is also necessary for a developer, however, to step out of the way to interact effectively as well.
Test often: This helps to avoid the build-up and crushing of morale from massive backlogs of issues. The safest method is known to be frequent checking.
Key takeaways
- Software Testing is a method of assessing a software application's performance to detect any software bugs.
- The software's real-world use will not be programmed, and without a guide, the testers need to break stuff.
A Software Testing Defect is a variation or deviation of the software programme from the demands or original business specifications of the end user. A software defect is a coding error that causes the results of a software programme that does not meet real specifications to be inaccurate or unexpected. While conducting the test cases, testers could come across certain defects.
Defect Prevention is much more efficient and effective in minimizing the number of defects and it is also very cost-effective during the early stage of the software process to repair the defects found. Most companies perform Defect Detection, Defect Elimination, and then Process Enhancement, which is collectively referred to as a Process for Defect Management.
Goals of Defect Management Process (DMP)
● Prevent the Defect
● Early Detection
● Minimize the impact
● Resolution of the Defect
● Process improvement
A defect management cycle contains the following stages
1) Discovery of Defect,
2) Defect Categorization
3) Fixing of Defect by developers
4) Verification by Testers,
5) Defect Closure
6) Defect Reports at the end of project
Fig 7: stages of defect management
Key takeaways
- A software defect is a coding error that causes the results of a software programme that does not meet real specifications to be inaccurate or unexpected.
- Defect Detection, Defect Elimination, and then Process Enhancement, which is collectively referred to as a Process for Defect Management.
- Defect Management is a systematic method to find and address bugs.
When the research team conducts the test cases, they come across a scenario where the real test result varies from the anticipated outcome. This variance is called a defect.
A Software Defect is simply a condition that does not satisfy the software requirement. The fault is a mistake or a flaw in the system that causes an unwanted or incorrect action.
The Defect Life Cycle ensures continuity and standardization of the process. In the life cycle, a defect crosses several states. It goes through different phases over its lifespan after a defect has been discovered, and it is generally known as the Defect Life Cycle.
In general, the defect life cycle begins from the point where the defect is detected or raised by the research team and ends when a defect is closed, either by ensuring that a production team does not duplicate it or reject it. From project to project, the number of states that a defect passes through varies.
Fig 8: defect life cycle
● Whenever the testing team detects a defect in the application, they lift the defect with the status as "NEW"
● If a new defect is tested by a QA lead and the defect is correct, "Open" will be the status of the defect and it is ready to be allocated to the development team.
● The status of the defect will be identified as "Assigned" if a QA lead assigns the defect to the corresponding developer.
● The developer rejects the defect when the developer thinks that the defect is not real or legitimate. The defect's status is labelled as "Rejected".
● If the observed defect is replicated twice or both of the identified defects have identical outcomes and replication steps, one defect status is changed to "Duplicate."
● If there are any difficulties or barriers to fixing a specific defect in the current release, then the defect will be taken instead of the current release in the future releases and then it will be listed as “Deferred” or “Postponed”
● If a developer is unable to reproduce the defect by the testing team's steps mentioned in “Steps to Reproduce” then the developer will label the defect as “Not Reproducible”
● If the developer is not sure about the reproduction steps given by a QA to reproduce the defect, he/she may mark it as "Need more data"
● If a defect is known and present in the manufacturing environment at the moment, the defect is identified as a "known defect."
● When the required improvements are made by a developer, the defect is marked as "Fixed."
● The developer now passes the defect to the testing team to check, so "Ready for Retest" changes the status of the developer.
● If there are no more problems with the defect and it is correctly verified, then the tester labels the defect as “Closed.”
● If the tester noticed that the defect was either reproducible or partly repaired, the defect would be labelled as "Reopened" when retesting the defect. The creator has to look into this defect again now.
Key takeaways
- The Defect Life Cycle ensures continuity and standardization of the process. In the life cycle, a defect crosses several states.
- The Defect Life Cycle ensures continuity and standardization of the process.
- In the life cycle, a defect crosses several states.
A Software Testing Bug Report is a systematic document regarding bugs found in a software programme. Every bug report contains information of bugs such as definition, date of bug discovery, name of the tester who discovered it, name of the developer who fixed it, etc. The bug report helps to detect similar vulnerabilities in the future in order to prevent them.
Your Bug Report should contain the following data when reporting the bug to the developer.
● Defect_ID - Unique identification number for the defect
● Defect Description - A thorough explanation of the defect, including descriptions of the module where the defect was discovered.
● Version - The version of the application where the defect was observed.
● Steps - Detailed steps and screenshots with which the flaws can be replicated by the developer.
● Date Raised - Date when the defect is raised
● Reference - Provide access to documents such as . Where in you Requirements, configuration, architecture or even even screenshots of the mistake to better explain the flaw.
● Detected By - The tester's name/ID that raised the defect
● Status - Defect status, more on this later.
● Fixed by - The developer's name/ID who repaired it
● Date Closed - The date at which the defect closes
● Severity - Describing the effect on the application of the defect
● Priority - Which is linked to the urgency of defect fixing. Based on the effect of urgency at which the defect should be corrected, the severity priority may be high/medium/low.
Key takeaways
- A Software Testing Bug Report is a systematic document regarding bugs found in a software programme.
- The bug report helps to detect similar vulnerabilities in the future in order to prevent them.
Debugging is the method of repairing a bug in the programme in the sense of software engineering. In other words, it applies to error detection, examination and elimination. This operation starts after the programme fails to function properly and ends by fixing the issue and checking the software successfully. As errors need to be fixed at all levels of debugging, it is considered to be an extremely complex and repetitive process.
Debugging process
Steps that are involved in debugging include:
● Identification of issue and preparation of report.
● Assigning the software engineer's report to the defect to verify that it is true.
● Defect Detection using modelling, documentation, candidate defect finding and checking, etc.
● Defect Resolution by having the device modifications required.
● Corrections validation.
Debugging strategies -
- To grasp the system, research the system for the longer term. Depending on the need, it helps debuggers create various representations of systems to be debugged. System analysis is also carried out actively to detect recent improvements made to the programme.
- Backward analysis of the issue that includes monitoring the software backwards from the fault message position to locate the region of defective code. A thorough region study is conducted to determine the cause of defects.
- Forward programme analysis includes monitoring the programme forward using breakpoints or print statements and studying the outcomes at various points in the programme. In order to locate the flaw, the region where the wrong outputs are obtained is the region that needs to be centered.
- Using previous software debugging experience, the software has similar problems in nature. This approach's effectiveness depends on the debugger's expertise.
Key takeaways
- Debugging is the method of repairing a bug in the programme in the sense of software engineering.
- it applies to error detection, examination and elimination.
References
- Roger Pressman, “Software Engineering: A Practitioner’s Approach”, McGraw Hill, ISBN 0-07-337597-7
- Ian Sommerville, “Software Engineering”, Addison and Wesley, ISBN 0-13-7035152
- Joseph Phillips, “IT Project Management-On Track From start to Finish”, Tata McGraw-Hill, ISBN13:978-0-07106727-0, ISBN-10:0-07-106727-2
- Pankaj Jalote, “Software Engineering: A Precise Approach”, Wiley India, ISBN: 9788-1265-2311-5
- Marchewka, “Information Technology Project Management”, Willey India, ISBN: 9788-1265-4394-6
- Rajib Mall, “Fundamentals of Software Engineering”, Prentice Hall India, ISBN-13:9788-1203-4898-1
