Monday, February 22, 2016

Why every software product needs quality assurance engineers

I've come across a fair number of people recently claiming that having a team of quality assurance engineers is superfluous, and that if the engineers were more careful, it wouldn't be necessary for test engineers.

Furthermore, the argument is that product managers need to specify how the product is both expected and not expected to work because software engineers will make mistakes if they don't understand the product they are solving for.

While that might be true for very simple code, it's unrealistic for more complicated products.

Recently, a car manufacturer recalled many of their cars because the hood latch can come loose while the car is in motion. So let's put this to the test, assume a test engineer would have caught it, and look at what happens when the product isn't tested properly.

Product manager:  the car shall have primary and secondary latches to keep the hood closed.  The primary latch will be operated from the inside, and the secondary will be a failsafe and strong enough to ensure that if the primary is accidentally engaged or fails, the hood will not open and blind the driver while the vehicle is in use. It can be opened from outside the vehicle after the primary is disengaged.

Latch developer:  I'll build two latches of strong material. It should be enough.

Test engineer:  I will test the prototype under the most extreme, windy conditions I can think of. Based on how the latches were engineered, I think it is most likely to fail if jarred, if there is sufficient wind putting stress on the latches, or the owner may believe the hood is closed when it actually is not completely closed.  I will use wind tunnel, off road simulations, and hood drop tests to simulate primary and secondary latch failures.

Product Manager, User Acceptance Testing:  As the business stakeholder, I'll make sure that the product does what is expected, enough that the product solves the business need (eg resulting in purchase).

However without the test engineer in the picture this becomes a blame game when customers thought they closed the hood but didn't close it all the way, especially if the average reasonable person thinks the hood "appears to be" closed.

This degrades into a blame game.  The latch developer is blamed for creating a latch that may not always close properly.  The product manager is blamed for not specifying how the customer would operate the latch, and says the developer should have picked a design that would account for something obvious like being able to easily operate the latch. And so the next time the product manager tries to be much more specific, defining not the problem to be solved but how the solution should be built, restricting the developer from choosing a better solution, and things just get worse from there.  Product managers writing functional stress tests instead of focusing on user acceptance, "will the customer buy/use it" tests.  Developers prioritizing scope over product quality. No one takes ownership of the quality, so the quality degrades without the proper expertise to examine and test the solution in the context of how this specific solution will really be used.

A simple part becomes a recall, costing millions of dollars to repair.

I'm not saying that's what happened for the car recently mentioned in the news, but if you followed my example, you'll realize that the product manager and the latch developer both did their job. The product manager defined the problem to be solved, and the developer created a solution. 

Without someone responsible for quality control, no one will realize there is a problem with the solution that was chosen. The product manager isn't technical enough to know how this implementation could fail, and the latch developer isn't aware that when the latch is attached to a hood, the "hood with latch" might not work as intended.

So yes, quality assurance engineers and test engineers, we need you. We might take you for granted at times, but bad things happen without you - that's when we realize your value.

No comments:

Post a Comment