Why It Matters (Test Engineering pt. 2)
Last week I wrote an article about the difference between Test Engineering and Quality Assurance.
Why does that distinction even matter beyond the title that is assigned to a department at a company? There are three reasons that this is a surprisingly destructive mindset:
- Cannot fix the quality problem if looking in the wrong places.
- Cannot test quality into software.
- Employee morale.
Cannot Fix Quality Problem by Making the Wrong Changes
If we have a problem with the quality of our product, we should want to fix that. If the business believes that quality is assured solely by those that are testing the software, then the quality problem will not ever be fixed.
In software development, whether in an agile or a waterfall environment, much of testing depends on hands-on execution of workflows. To be able to execute a workflow, there must be a unit of software in some state of completion to test. Something needs to have been developed.
As I discussed in Part 1, a test engineer can only inform the team that they found an issue. It is up one or more others whether that issue is fixed.
Many times, then, when a low quality product is released the test engineering team knows a clunker is going out into the world. Date-driven release cycles and slipping development completion dates drive the product out the door. Go on forums and one of the first questions you will often see is “did the QA team even test this?”
They probably did, and are probably just as frustrated about the decision to not fix that bug as the end user is. Quality cannot be assured without each member of the engineering and delivery team being dedicated to releasing a quality product.
Often quality issues arise because of date-driven development and release cycles. The software has a release date of 3 weeks from now, which means we have two weeks to develop it and a week to test. But then development slips and it takes 13 of those 15 days to reach code completion. Now there are only two days to execute 5 days’ worth of tests. In those cases the testing team is often blamed for holding up the release because testing is taking too long. This is unjust and puts test engineers in a spot where they receive blame either way as soon as the dev time slips, and it almost always slips.
Now this product that turned out to be more complex than anticipated resulted in additional dev time and reduced test time. It goes out the door and is of low quality. The QA team is the first to come under scrutiny because they’re supposed to catch those bugs. Didn’t they even test this?
Why weren’t these bugs caught before the product was released (Test Engineering)?
Why were there so many bugs in the first place (Software Engineering)?
Why was the timeline so aggressive that sufficient testing could not occur (Business)?
Who signed off on the test cases that were created if they were insufficient for coverage (Product Management)?
By making the testing team the sole owners of quality by calling them Quality Assurance, we can never fix the quality problems with the product. A product that has severe quality issues is the sum of its low quality parts and indicative of problems with every team associated with its release.
You Cannot Test Quality into Software
A lot of test engineers get weird about agreeing that a piece of software is “good enough”. We will never find all of the defects in a product, and even if we did they will never all get fixed. That is unrealistic.
If it was possible to test the quality into software I guarantee that test engineers everywhere would rejoice because they could just test the issues away. Unfortunately we can’t do that. But when we assign the testing team a moniker that implies that they are the sole owners of product quality, then that expectation can be set.
Finding bugs is only useful when they are triaged and fixed quickly to produce a high-quality experience for the end user. Having hundreds or thousands of open defects is a lot of wasted effort and an awful user experience. The bugs were caught; test engineering did their job, but the issues were never resolved. You cannot test quality into software.
Employee Morale
Being blamed for something that is not your fault is a terrible feeling. When this blame is related to your work, it can be very stressful. Any uncertainty in the future of your employment introduces impediments to your work.
Over time, this really wears down a team. Even if a job isn’t on the line, losing credibility as a team is detrimental.
A negative feedback loop can develop to varying degrees of intensity:
Poor software is released -> “QA” is blamed.
Next increment of software is being developed -> “QA” doesn’t catch stuff anyway so let’s release.
Poor software is released -> “QA” is blamed.
This is a very very oversimplified example whose primary intent is to show how hurt credibility of a test engineering organization can increasingly hurt the quality of software.
Now that team whose mission is to inform the organization of issues they have identified in the software sees worse and worse software going out the door. At least some of the blame falls onto them for a cycle which is increasingly out of their control.
It is not good for employee morale.
Conclusion
Every department in the software development lifecycle has an important role to play. This post is not intended to dunk on “development for taking too long” or “product for being date-driven”. Time compression and the need to release are realities of software development. That is a fact.
Test engineering plays a very important role and can be very effective when utilized appropriately, and the name we give that organization can have a surprising impact.