What is QA documentation, and how to reduce its creation and maintenance costs?

Andrii Hilov, QA Team Lead at ITRex, has written another article discussing quality assurance challenges and pitfalls in software projects. This time, Andrii delves into QA documentation and the role it plays in developing high-performance software — on time, on budget, and in line with your business goals.

Here’s what he has to say about it.

As a QA Team Lead at an enterprise software development company ITRex, I’m perfectly aware of our client’s aspirations to reduce software development costs while launching a fully functioning product on time and to maximum value.

While these goals are understandable, I advise against dismissing your QA team early in the project, even if they don’t find bugs on a daily basis, although this might seem an easy option to cut the paycheck and speed up software release cycles.

Also, I recommend you follow quality assurance best practices throughout the project to validate that your solution and all of its features function as expected and do not compromise your cybersecurity.

And one of such practices is creating and maintaining proper QA documentation.

What is quality assurance documentation exactly? How can it help you reap the most benefit from tapping into QA and testing services? And is there a way to optimize the costs and effort associated with preparing QA documentation while minimizing the risk of developing a poorly architected, bug-ridden application and having to rebuild the whole thing from the ground up?

Let’s find that out!

Introduction to QA documentation

QA documentation is a collection of documents and artifacts created and maintained by a quality assurance team during the software development and testing process.

It may include various documents that outline the testing strategy, test plans, test cases, test scripts, test data, test logs, bug reports, and any other documentation related to the QA activities. These documents facilitate communication among QA team members, provide guidelines for testing, and help in identifying and resolving issues efficiently.

Thus, QA documentation plays a vital role in ensuring the quality and reliability of software products — and that’s the major objective our clients pursue.

What QA documents are used in software projects?

For this article’s purpose, we’ll give you a brief overview of quality assurance documents that form the backbone of testing documentation in a software development project:

• A test plan is a QA document that outlines the overall approach, goals, scope, resources, and schedule of software testing activities. Simply put, it covers:
  1. The name and description of a project, including the types of apps under testing and their core functionality
  2. The preferred testing methods (manual, automated, mixed) and test types (new features, integrations, compatibility, regression, etc.)
  3. The features that need to be tested, alongside an approximate schedule for each testing activity
  4. Optimum team composition
  5. An overview of risks and issues that might arise during the testing process
  6. A list of testing documents that your QA team will use during the project

A rule of thumb is to write a test plan at the beginning of a software project when your IT team defines functional and non-functional requirements for a software solution, chooses an appropriate technology stack and project management methodology, and creates a project roadmap.

It normally takes up to three days to put up and review a simple test plan without test cases.

• Test cases describe specific test scenarios, including the input data, expected results, and steps to execute. Test cases are designed to verify the functionality, performance, or other aspects of a software product. Please note that test cases are used by both manual testing services and QA automation services teams. This way, you’ll ensure maximum test coverage, meaning no bugs will manifest themselves in production code.

Even though a skilled QA engineer could write a high-level test case in just ten minutes, the number of test cases for a medium-sized project could easily exceed 4,000 (and counting). Multiply that number by the average middle QA engineer hourly rate ($65 per man hour for the North American market), and you’ll arrive at an impressive figure.

• Checklists are concise, itemized lists of actions or tasks that need to be completed or verified during the testing process. Thus, a checklist in QA documentation usually includes a complete rundown of functional modules, sections, pages, and other elements of an app or cyber-physical system that require a QA team’s attention.

In smaller projects, checklists can successfully replace detailed test cases (more on that later.)

• Test scripts are chunks of code written using specific testing tools or frameworks, such as Selenium, Appium, and Cucumber. These scripts automate the execution of test cases, making the testing process more efficient — specifically, in large and complex software projects like multi-tenant SaaS systems and popular B2C apps, which are updated frequently and where even the smallest bugs may negatively impact user experience.
• Test data is the data used by QA engineers to assess the performance, functionality, reliability, and security of a software solution under various conditions. It may include sample input values, boundary conditions, and various scenarios. For instance, your QA team may use positive and negative test data to validate that only correct login credentials may be used for entering a software system. Similarly, test data can be used for implementing age restrictions in certain types of apps or investigating how an application handles increased workloads.

• Test logs document the test execution process, including the date and time of test performance, the summary of the executed test cases, the results your QA team achieved, screenshots, and any issues or observations noted during testing. A test log is a vital source of information for tracking the testing progress, identifying patterns or trends in test results, and providing a historical record of the testing activities. It helps identify and resolve issues efficiently and serves as a reference for future testing efforts or audits.
• Defect or bug reports are testing documents that detail defects and issues found during QA activities. Specifically, they describe the detected bugs, their severity and priority, and the conditions under which the defects occur. A QA manager uses bug reports to assign tasks to software testing specialists and track their status.

• A traceability matrix maps the relationship between test cases and requirements or other artifacts. It helps ensure that all requirements are adequately covered by test cases, allows for tracking the test coverage across the project, and eliminates redundant testing activities.
• A test completion report summarizes the testing activities performed in a project, including the test execution status, the number of test cases executed, defects found, and any pending tasks.

Why is QA documentation important?

Having quality assurance documentation helps attain the exact results that the customer and the software engineering team expect.

This is achieved by a combination of factors, including the following:

1. QA documentation provides clear instructions and guidelines that software testing specialists can follow to perform tasks consistently, reducing variations and improving the overall quality of products or services.
2. Quality assurance documentation reduces the likelihood of detecting critical defects and errors in software solutions late in the development process, thus playing a pivotal role in budget control. QA experts suggest that the cost of fixing bugs increases exponentially with every project stage, ranging from 3X for the design/architecture phase to 30X and more for the deployment phase.
3. Quality assurance documentation helps ensure compliance with the regulatory requirements and standards your organization must meet by simplifying audits and providing evidence of established processes, procedures, and quality controls.
4. By documenting procedures, controls, and risk assessment processes, software testing documentation helps organizations identify potential risks and take preventive measures to minimize their impact on their business and customer satisfaction.
5. New hires can refer to your QA documentation to understand the quality processes and procedures in a software project, reducing the learning curve and ensuring consistent training across the organization.
6. By documenting non-conformances, corrective actions, and lessons learned, companies can identify areas for improvement and implement changes to enhance efficiency and quality.
7. Having well-documented QA processes and procedures can enhance customer confidence in your company’s products or services. Extensive software testing documentation demonstrates a commitment to quality and assures that the organization has robust systems in place to deliver consistent and reliable results.
8. In situations where legal disputes or product recalls arise, QA documentation can serve as important evidence. It can demonstrate that your organization has followed established quality processes, taken necessary precautions, and fulfilled its obligations.

How long does it take to create QA documentation?

An honest answer to this question will be, “It depends.”

Specifically, the timeframe and the associated costs depend on several factors, such as the size of your organization and the complexity of its processes, the industry you’re in, and the type of software you’re building.

If you’ve previously embarked on software development projects and have an in-house QA team, you might be able to reuse existing QA documentation for new projects. Using templates and specialized tools for creating and maintaining software testing documentation, such as project management and wiki software, is helpful, too.

Do you always need QA documentation — and is it possible to reduce its creation and maintenance costs?

However useful, quality assurance documentation may increase software project costs due to the additional effort and personnel required for its creation and maintenance.

This might be an issue for startups operating on a shoestring or enterprises undergoing digital transformation in times of recession.

Does every type of software project need super-detailed QA documentation then — and is it possible to reduce the costs associated with it?

To determine the best approach to QA document creation, consider the following factors:

• Project size and budget. In the case of small-budget and short-term projects (unless we talk about highly innovative and technical projects executed by large IT teams), there is no need to overcomplicate the documentation process, so your QA squad can opt for checklists instead of detailed test cases. Regarding the test plan document, which determines the overall testing strategy, we can also forgo writing it in cases where there is no budget for it or if the project is short-term and does not involve leading-edge technologies.
• QA team size and experience. The more QA engineers on the project and the less experience they have in quality assurance, the more challenging it is to control the testing process. Therefore, you need extensive quality assurance documentation to keep the team members on the same page. In such cases, it is advisable to lean towards test cases rather than checklists to more effectively distribute tasks among engineers based on their experience and knowledge, and to involve more experienced QA specialists, who normally have higher hourly rates, in test case creation.
• Agile vs. Waterfall approach to project management. While the ITRex team has summarized the key differences between Agile and Waterfall methodologies in this blog post, it’s worth mentioning what sets the two approaches apart in terms of quality assurance. In Waterfall, software testing is saved for last, meaning your QA team will conduct tests only when the coding part is 100% complete. For obvious reasons, they can’t do it without proper quality assurance documentation, which should be prepared during the requirements elicitation phase. In Agile, where IT teams tend to build smaller pieces of software iteratively and test the code at the end of each cycle, creative comprehensive QA documentation beforehand is not preferred. Still, I recommend you write a test plan to better align the current situation with the customer’s and software engineers’ expectations.

Overall, having QA documentation could benefit any software development project, no matter the complexity and size.

As a client-oriented company, however, we’re always ready to suggest workarounds considering your objectives and budget.