• 11 Step Software Testing Process
• Software Testing Step By Step
• Christine Lakin

Manual Testing is a process of finding out the defects or bugs in a software program. In this method the tester plays an important role of end user and verifies that all the features of the application are working correctly. Dec 03, 2013  Step 2: Develop the Test Plan. Test plan describes how testing will be accomplished. Objective of a test plan is to describe all testing that is to be accomplished, together with the resources and schedule necessary for completion. Should provide background information on the software being tested, test objectives and risks,.

• Software Testing Tutorial

• Software Testing Useful Resources

• Selected Reading

What is Testing?

Review test types and fill in the holes 5. Write a test case for each test require-ment 6. Group test cases into test scripts www.stqemagazine.com Software Testing & Quality Engineering September/October 2000 42 Testing STEP-BY-STEP QUICK LOOK A simple method for creating test cases Using brainstorming and tables to produce test scripts.

Testing is the process of evaluating a system or its component(s) with the intent to find whether it satisfies the specified requirements or not. In simple words, testing is executing a system in order to identify any gaps, errors, or missing requirements in contrary to the actual requirements.

INTEGRATION TESTING is a level of software testing where individual units are combined and tested as a group. The purpose of this level of testing is to expose faults in the interaction between integrated units. Test drivers and test stubs are used to assist in Integration Testing. Definition by ISTQB integration testing: Testing performed to expose. Step by step guide to UFT QTP Automation FrameworkPart 1 Part 2 Part 3 Part 4 Part 5 Part 6 Part 7 The logic for the first test for this tutorial is:If thePage link exists, then click the link. POSTMAN API Testing - Step by Step for Beginners 4.3 (826 ratings) Course Ratings are calculated from individual students’ ratings and a variety of other signals, like age of rating and reliability, to ensure that they reflect course quality fairly and accurately.

According to ANSI/IEEE 1059 standard, Testing can be defined as - A process of analyzing a software item to detect the differences between existing and required conditions (that is defects/errors/bugs) and to evaluate the features of the software item.

Who does Testing?

It depends on the process and the associated stakeholders of the project(s). In the IT industry, large companies have a team with responsibilities to evaluate the developed software in context of the given requirements. Moreover, developers also conduct testing which is called Unit Testing. In most cases, the following professionals are involved in testing a system within their respective capacities −

• Software Tester
• Software Developer
• Project Lead/Manager
• End User

Different companies have different designations for people who test the software on the basis of their experience and knowledge such as Software Tester, Software Quality Assurance Engineer, QA Analyst, etc.

It is not possible to test the software at any time during its cycle. The next two sections state when testing should be started and when to end it during the SDLC.

When to Start Testing?

An early start to testing reduces the cost and time to rework and produce error-free software that is delivered to the client. However in Software Development Life Cycle (SDLC), testing can be started from the Requirements Gathering phase and continued till the deployment of the software.

It also depends on the development model that is being used. For example, in the Waterfall model, formal testing is conducted in the testing phase; but in the incremental model, testing is performed at the end of every increment/iteration and the whole application is tested at the end.

Testing is done in different forms at every phase of SDLC −

• During the requirement gathering phase, the analysis and verification of requirements are also considered as testing.

• Reviewing the design in the design phase with the intent to improve the design is also considered as testing.

• Testing performed by a developer on completion of the code is also categorized as testing.

When to Stop Testing?

It is difficult to determine when to stop testing, as testing is a never-ending process and no one can claim that a software is 100% tested. The following aspects are to be considered for stopping the testing process −

• Testing Deadlines

• Completion of test case execution

• Completion of functional and code coverage to a certain point

• Bug rate falls below a certain level and no high-priority bugs are identified

• Management decision

Verification & Validation

These two terms are very confusing for most people, who use them interchangeably. The following table highlights the differences between verification and validation.

Given below are some of the most common myths about software testing.

Myth 1: Testing is Too Expensive

Reality − There is a saying, pay less for testing during software development or pay more for maintenance or correction later. Early testing saves both time and cost in many aspects, however reducing the cost without testing may result in improper design of a software application rendering the product useless.

Myth 2: Testing is Time-Consuming

Reality − During the SDLC phases, testing is never a time-consuming process. However diagnosing and fixing the errors identified during proper testing is a time-consuming but productive activity.

Myth 3: Only Fully Developed Products are Tested

Reality − No doubt, testing depends on the source code but reviewing requirements and developing test cases is independent from the developed code. However iterative or incremental approach as a development life cycle model may reduce the dependency of testing on the fully developed software.

Myth 4: Complete Testing is Possible

Reality − It becomes an issue when a client or tester thinks that complete testing is possible. It is possible that all paths have been tested by the team but occurrence of complete testing is never possible. There might be some scenarios that are never executed by the test team or the client during the software development life cycle and may be executed once the project has been deployed.

Myth 5: A Tested Software is Bug-Free

Reality − This is a very common myth that the clients, project managers, and the management team believes in. No one can claim with absolute certainty that a software application is 100% bug-free even if a tester with superb testing skills has tested the application.

Myth 6: Missed Defects are due to Testers

Reality − It is not a correct approach to blame testers for bugs that remain in the application even after testing has been performed. This myth relates to Time, Cost, and Requirements changing Constraints. However the test strategy may also result in bugs being missed by the testing team.

Myth 7: Testers are Responsible for Quality of Product

Reality − It is a very common misinterpretation that only testers or the testing team should be responsible for product quality. Testers’ responsibilities include the identification of bugs to the stakeholders and then it is their decision whether they will fix the bug or release the software. Releasing the software at the time puts more pressure on the testers, as they will be blamed for any error.

Myth 8: Test Automation should be used wherever possible to Reduce Time

Reality − Yes, it is true that Test Automation reduces the testing time, but it is not possible to start test automation at any time during software development. Test automaton should be started when the software has been manually tested and is stable to some extent. Moreover, test automation can never be used if requirements keep changing.

Myth 9: Anyone can Test a Software Application

Reality − People outside the IT industry think and even believe that anyone can test a software and testing is not a creative job. However testers know very well that this is a myth. Thinking alternative scenarios, try to crash a software with the intent to explore potential bugs is not possible for the person who developed it.

Myth 10: A Tester's only Task is to Find Bugs

Reality − Finding bugs in a software is the task of the testers, but at the same time, they are domain experts of the particular software. Developers are only responsible for the specific component or area that is assigned to them but testers understand the overall workings of the software, what the dependencies are, and the impacts of one module on another module.

Testing, Quality Assurance,and Quality Control

Most people get confused when it comes to pin down the differences among Quality Assurance, Quality Control, and Testing. Although they are interrelated and to some extent, they can be considered as same activities, but there exist distinguishing points that set them apart. The following table lists the points that differentiate QA, QC, and Testing.

Audit and Inspection

Audit − It is a systematic process to determine how the actual testing process is conducted within an organization or a team. Generally, it is an independent examination of processes involved during the testing of a software. As per IEEE, it is a review of documented processes that organizations implement and follow. Types of audit include Legal Compliance Audit, Internal Audit, and System Audit.

Inspection − It is a formal technique that involves formal or informal technical reviews of any artifact by identifying any error or gap. As per IEEE94, inspection is a formal evaluation technique in which software requirements, designs, or codes are examined in detail by a person or a group other than the author to detect faults, violations of development standards, and other problems.

Formal inspection meetings may include the following processes: Planning, Overview Preparation, Inspection Meeting, Rework, and Follow-up.

Testing and Debugging

Testing − It involves identifying bug/error/defect in a software without correcting it. Normally professionals with a quality assurance background are involved in bugs identification. Testing is performed in the testing phase.

Debugging − It involves identifying, isolating, and fixing the problems/bugs. Developers who code the software conduct debugging upon encountering an error in the code. Debugging is a part of White Box Testing or Unit Testing. Debugging can be performed in the development phase while conducting Unit Testing or in phases while fixing the reported bugs.

Many organizations around the globe develop and implement different standards to improve the quality needs of their software. This chapter briefly describes some of the widely used standards related to Quality Assurance and Testing.

ISO/IEC 9126

This standard deals with the following aspects to determine the quality of a software application −

• Quality model
• External metrics
• Internal metrics
• Quality in use metrics

This standard presents some set of quality attributes for any software such as −

• Functionality
• Reliability
• Usability
• Efficiency
• Maintainability
• Portability

The above-mentioned quality attributes are further divided into sub-factors, which you can study when you study the standard in detail.

ISO/IEC 9241-11

Part 11 of this standard deals with the extent to which a product can be used by specified users to achieve specified goals with Effectiveness, Efficiency and Satisfaction in a specified context of use.

This standard proposed a framework that describes the usability components and the relationship between them. In this standard, the usability is considered in terms of user performance and satisfaction. According to ISO 9241-11, usability depends on context of use and the level of usability will change as the context changes.

ISO/IEC 25000:2005

ISO/IEC 25000:2005 is commonly known as the standard that provides the guidelines for Software Quality Requirements and Evaluation (SQuaRE). This standard helps in organizing and enhancing the process related to software quality requirements and their evaluations. In reality, ISO-25000 replaces the two old ISO standards, i.e. ISO-9126 and ISO-14598.

SQuaRE is divided into sub-parts such as −

• ISO 2500n − Quality Management Division
• ISO 2501n − Quality Model Division
• ISO 2502n − Quality Measurement Division
• ISO 2503n − Quality Requirements Division
• ISO 2504n − Quality Evaluation Division

The main contents of SQuaRE are −

• Terms and definitions
• Reference Models
• General guide
• Individual division guides
• Standard related to Requirement Engineering (i.e. specification, planning, measurement and evaluation process)

ISO/IEC 12119

This standard deals with software packages delivered to the client. It does not focus or deal with the clients’ production process. The main contents are related to the following items −

• Set of requirements for software packages.
• Instructions for testing a delivered software package against the specified requirements.

Miscellaneous

Some of the other standards related to QA and Testing processes are mentioned below −

This section describes the different types of testing that may be used to test a software during SDLC.

Manual Testing

Manual testing includes testing a software manually, i.e., without using any automated tool or any script. In this type, the tester takes over the role of an end-user and tests the software to identify any unexpected behavior or bug. There are different stages for manual testing such as unit testing, integration testing, system testing, and user acceptance testing.

Testers use test plans, test cases, or test scenarios to test a software to ensure the completeness of testing. Manual testing also includes exploratory testing, as testers explore the software to identify errors in it.

Automation Testing

Automation testing, which is also known as Test Automation, is when the tester writes scripts and uses another software to test the product. This process involves automation of a manual process. Automation Testing is used to re-run the test scenarios that were performed manually, quickly, and repeatedly.

Apart from regression testing, automation testing is also used to test the application from load, performance, and stress point of view. It increases the test coverage, improves accuracy, and saves time and money in comparison to manual testing.

What to Automate?

It is not possible to automate everything in a software. The areas at which a user can make transactions such as the login form or registration forms, any area where large number of users can access the software simultaneously should be automated.

Furthermore, all GUI items, connections with databases, field validations, etc. can be efficiently tested by automating the manual process.

When to Automate?

Test Automation should be used by considering the following aspects of a software −

• Large and critical projects
• Projects that require testing the same areas frequently
• Requirements not changing frequently
• Accessing the application for load and performance with many virtual users
• Stable software with respect to manual testing
• Availability of time

How to Automate?

Automation is done by using a supportive computer language like VB scripting and an automated software application. There are many tools available that can be used to write automation scripts. Before mentioning the tools, let us identify the process that can be used to automate the testing process −

• Identifying areas within a software for automation
• Selection of appropriate tool for test automation
• Writing test scripts
• Development of test suits
• Execution of scripts
• Create result reports
• Identify any potential bug or performance issues

Software Testing Tools

The following tools can be used for automation testing −

• HP Quick Test Professional
• Selenium
• IBM Rational Functional Tester
• SilkTest
• TestComplete
• Testing Anywhere
• WinRunner
• LoadRunner
• Visual Studio Test Professional
• WATIR

There are different methods that can be used for software testing. This chapter briefly describes the methods available.

Black-Box Testing

The technique of testing without having any knowledge of the interior workings of the application is called black-box testing. The tester is oblivious to the system architecture and does not have access to the source code. Typically, while performing a black-box test, a tester will interact with the system's user interface by providing inputs and examining outputs without knowing how and where the inputs are worked upon.

The following table lists the advantages and disadvantages of black-box testing.

White-Box Testing

White-box testing is the detailed investigation of internal logic and structure of the code. White-box testing is also called glass testing or open-box testing. In order to perform white-box testing on an application, a tester needs to know the internal workings of the code.

The tester needs to have a look inside the source code and find out which unit/chunk of the code is behaving inappropriately.

The following table lists the advantages and disadvantages of white-box testing.

Grey-Box Testing

Grey-box testing is a technique to test the application with having a limited knowledge of the internal workings of an application. In software testing, the phrase the more you know, the better carries a lot of weight while testing an application.

11 Step Software Testing Process

Mastering the domain of a system always gives the tester an edge over someone with limited domain knowledge. Unlike black-box testing, where the tester only tests the application's user interface; in grey-box testing, the tester has access to design documents and the database. Having this knowledge, a tester can prepare better test data and test scenarios while making a test plan.

A Comparison of Testing Methods

The following table lists the points that differentiate black-box testing, grey-box testing, and white-box testing.

In the previous post of this series around Manual Testing, I tried to throw as much light as possible on the basics of Manual Testing.

If you missed it, you can read it here.

I hope it was successful in taking you as close as possible to the answers you were looking for.

In that case, wouldn’t you love to know more about the practical implementation of Manual Testing, how to get more familiar with it and how to actually start a career in it?

This article will throw light on all these aspects.

Let’s start.

What You Will Learn:

Manual Testing Cycle

To understand Manual Testing Cycle or Software Test Life Cycle (STLC), first of all, we need to understand Software Development Life Cycle (SDLC), which I am sure you already have an understanding of.

People refer to them separately but not sure if they can really co-exist. They are that tightly integrated with each other. Well, even these cycles have so many versions of them created and floating in the internet space, they vary majorly on which development model is selected.

As most of the world is going Agile these days, I will keep my stuff simplified around Agile.

7 Practical Manual Testing Steps Before Production Release

Here I go.

Remember I am talking about both SDLC and STLC.

#1) Requirement Gathering

Business Analyst (Person/Team responsible for Requirement Gathering) documents the requirements. They document the requirements, that’s the highlight, you can keep the focus on that only. Where it is documented matters less.

People use anything to document these which suits them but not limited to traditional platforms like MS word doc, modern platforms like Jira/Rally and new age tools like Trello.

#2) Requirement Discussion/Sharing

Business Analyst is then supposed to share documented requirements with Development Team, Testing Team, and the UX team (if needed). This usually happens via a formal meeting where SPOCs (Single Point Of Contacts or an entire team, it depends) from all three functions meet and understand the entire requirement.

In a healthy work culture, requirements get discussed from every angle and each member of the meeting can ask questions and doubts. Once all the questions are answered and needed modification in the requirement is done, this phase can be considered as Done. Again what one calls this particular meeting/step and its documentation differ company to company.

Further reading => How to Review SRS Document

Once all the questions are answered and needed modifications in the requirement are done, this phase can be considered as Done.

Again what one calls this particular meeting/step and its documentation differ company to company.

For example, the documentation is called or break down as SRS (System Requirement Specification), Requirement Document, Epic, User Story, Story point (possibly, smallest requirement unit), etc. On the similar lines, this meeting in which requirement is shared is called as Requirement Discussion meeting, Grooming, Hole-punching meeting, etc., I hope you get my point?

Pressing on these terminologies so that you always remember the main idea irrespective of the different names.

Post this meeting two steps gets triggered at the same time, in no particular order, refer next two steps.

#3) Designing

Development team starts with their technical designing as soon as the requirements are discussed and when there are no major pending points. What is done in this phase again differs company to company.

This phase may involve but not limited to the following tasks:

• Deciding the development approach
• Preparing design document
• Designing the flow charts
• Estimating the efforts
• Figuring out impact of this new requirement on any existing functionality
• Need to patch existing data, etc.

The UX team may also get involved in this phase when there is an UI change or new screen is to be developed. The UX team helps Development team and Testing team with UI prototype for the functionality/feature in the discussion. This can be a Photoshop document, simple image, PowerPoint presentation or anything else which will make development team understand how the screens should be developed.

Note:Ideally these screens or at least their draft versions, are shown in Requirement discussion session only to help the team build a better understanding. It gets tagged to original requirement so that it can be referred to at any given time.

#4) Test Scenario/Test Case designing

Parallel to the Designing phase, the Testing team starts with building Test scenarios and/or Test cases based on discussed requirements. Whether Test scenarios are always written first and then break into Test cases is something which is again not constant.

Software Testing Step By Step

In my opinion, whether you document the Test scenarios or not, they are always there before Test cases. Test Scenario is your bullet point you can say, they guide you to drill down things further. Once the test case writing is over, it can be shared with Development team to give them an idea of the Testing scope and they can also make sure that development which has happened or happening is satisfying the written test cases.

Once the test case writing is over, it can be shared with Development team to give them an idea of the Testing scope and they can also make sure that development which has happened or happening is satisfying the written test cases.

Test cases once written ideally get reviewed by a Test Lead or peer from many angles like:

• Requirement coverage
• Spelling grammar
• Test case writing standards(nothing but a template which a team/company follows)
• Backward compatibility
• Platform compatibility
• Test data references
• Types of testing targeted, etc.

Christine Lakin

Further reading => Writing test cases from SRS document

Ideally, only after review and needed modification, they are passed on to the Development team.

When I said ‘once Test case writing is over’, I meant once ‘all the test cases are written based on complete knowledge of given requirement and possible test scenarios uncovered till that particular time’. It is near impossible to have 100% test case coverage on the first go.

There will be defects which you will find in random (but intended) actions, in purely random actions (monkey testing) and in some rare scenarios. There are chances you will miss on few of these. And at some time you might miss out even very basic ones, after all, you are human. But here, at least a good test case review and structured way of test case writing can save you.

More often than not, a tester or testing team keeps on adding more and more test cases to the existing chunk as they uncover the truth or think more about the requirements.

Well, by now some of you must be doubting my knowledge of Software Testing as some word (which has kind of become a norm in Software Testing) is not used by me yet. Test Plan right?

Let me say something on this. I believe strongly in the need for most of the information which is mentioned in the Test Plan, but documenting them all at the same place and making it absolute mandatory is something I find debatable.

Anyways, that’s altogether a separate topic to discuss. To share a ‘suits all’ information on this is difficult but let me try.

Either you, you with your test lead or your test lead prepare Test Plan or you document the required information at different places.

Further Reading => How to Write Test Plan Document

Information which should be absolutely frozen at this stage:

• The scope of Testing: Requirement, Backward Compatibility, Platforms, Devices, etc.
• Person/Team who is going to test
• Test effort estimation
• Limitations: Any assumptions made or limitations accepted in advance.
• People additionally document entry criteria, exit criteria, risk, etc. which I think doesn’t really need separate mention as it should be a normal understanding rather.
  • Entry Criteria (When to start testing): Few start when there is testable part of functionality available for testing. Few wait for entire functionality to be testable. Once the basic flow is found to be working, testing starts.
  • Exit Criteria (When to stop): When there are no blocker, critical and major (subjected to impact) defects in open stage testing can be stopped. Or in mid-way, when there are way too many defects being faced testing can be stopped by appropriate stakeholders.
  • Risk: Business risk or functional risk if testing doesn’t happen as per documented plan.

#5) Development phase

The development team after designing phase starts with actual development and unit testing as and when they are done with the development of testable requirement chunks. They can pass on the functionality for testing in chunks as and when it is implemented or they can pass entire functionality at once.

In an ideal scenario, formal code review and white box testing happen before passing on the developed functionality for Testing. ideally, the Development team should also refer to Test cases provided by the testing team in addition to requirements and design documents.

#6) Testing phase

As mentioned earlier, the start of this phase differs company to company, team to team.

The testing team starts testing either when testable (something which can be independently tested) part of the entire requirement is developed or when the entire requirement is developed.

Let me drill down further in this phase and talk about the important tasks:

• Tester/Testing Team starts with testing round (exploratory testing and execution of written test cases) and logging defects
• Development team resolves them as per priority.
• New build (code) is taken on environment on which testing is happening
• Resolved defects are then verified by Tester/Testing Team and marked as Fixed
• This cycle continues till the time exit criteria is reached.
• Please note that as needed, defects are also marked as Invalid, Duplicate and can also be categorized as Enhancements.

Another thing which differs company to company is how many testing rounds to be done. Like in some cases, the first round of testing happens on small feature parts as they are ready, followed by an end-to-end testing round on another environment once all requirements are developed. But again, I have also heard of people doing three proper full testing rounds and fourth as sanity/smoke round.

The first agenda behind doing multiple testing rounds is testing the functionality on different environments and second for doing end-to-end testing once all story points are developed. Sanity round usually happens to gain quick confidence once all stories in a release are developed and tested independently.

Read detailed steps => Test Execution phase

#7) Business Analyst (BA) Review

Business analyst reviews the asked functionality either by referring to test result or by referring to test result plus playing around with application to get an actual feel. This step again is subjected to different actions across companies.

The BA may review the scope of entire release in one go or in chunks. Depending on the same, this step might come before final sanity testing or after final sanity testing round by the testing team.

Above 7 steps happen for all the user stories/requirements to be fulfilled in particular Release (Shipment). Once all these steps are completed for all the requirements, the release is said to be ready for shipment.

#8) Shipment/Release

The release is tagged as Ready for Shipment post successful review by the Business Analyst.

Recommended read => Test release process

Types of Manual (read Human) Testing

Well if we have to talk about overall types of testing in numbers then somewhere I found over 100 types of testing with different names. To be honest I am not smart enough to understand the distinction between all of those types (pun intended).

It is straight and simple: Testing the functionality of the application against the given requirement with human efforts and intelligence. It gets further divided into few types based on scope and agenda of testing. Types listed in next para.

It gets further divided into few types based on scope and agenda of testing. Types listed in next para.

If I am allowed to, let me speak few lines of Human Testing (which I encourage every tester to do over just manual functional testing). Now don’t get confused, in my view manual functional testing is a subset of Human Testing. Because there are so many things that only Human mind can do.

Below is the list with some of the popular and important testing types which can be categorized into Human Testing:

• Manual Functional Testing: Testing the functionality of the application against the given requirement with human efforts and intelligence. Further gets divided into quite a few types based on scope and agenda of testing, like system testing, unit testing, smoke testing, sanity testing, integration testing, regression testing, UI Testing, etc.
• Performance Testing: This gets categorized into NonFunctional Testing right? But again it is the human who implements it, though execution gets done by either human or tool. The tester should at least do this testing in terms of response time (to see if it is acceptable) if he is not supposed to use any tool for load testing and all.
• Browser/Platform Compatibility Testing: Application under test should look and work as expected (obviously there can be a minor difference depending on browser engine) across browsers and platforms (or devices if it is a mobile application).
• Usability Testing: Let me agree first of all that this is a huge topic in itself and usually owned by specialists in usability testing. I still believe that as a tester we should at least report or highlight if we find something as less usable, or we should share our view.
• Security Testing: Again very huge testing type and requires a lot of practical knowledge of course. The tester should try to learn and execute at least basic tests like URL tampering, Cross-site scripting, SQL injection, Session hijacking, etc. depending on available knowledge and wherever applicable.
• Multi-tenancy Testing: If your application is multi-tenant, i.e. single instance holding data of multiple clients, then this testing is a must. Irrespective of explicit mention in requirements this should be done. One client’s data being shown to another is kind of a development and testing crime.

Note: Above views are my personal views. I also recommend you to have a look at the extensive list of testing types for your knowledge and follow/use them if you find it necessary. Over years I have understood that whether you use something or nor, you believe in something or not, you should still have some knowledge of widely used concepts in your field.

That’s all for this part. Thank you for reading. Do share your views/feedback in comments.

In next and last part of this manual testing tutorial's series, I will try to help you all with what preparation you should be doing if you are looking to get into the testing field and what are possible ways to get in there.