Monday, February 22, 2016

Why every software product needs quality assurance engineers

I've come across a fair number of people recently claiming that having a team of quality assurance engineers is superfluous, and that if the engineers were more careful, it wouldn't be necessary for test engineers.

Furthermore, the argument is that product managers need to specify how the product is both expected and not expected to work because software engineers will make mistakes if they don't understand the product they are solving for.

While that might be true for very simple code, it's unrealistic for more complicated products.

Recently, a car manufacturer recalled many of their cars because the hood latch can come loose while the car is in motion. So let's put this to the test, assume a test engineer would have caught it, and look at what happens when the product isn't tested properly.

Product manager:  the car shall have primary and secondary latches to keep the hood closed.  The primary latch will be operated from the inside, and the secondary will be a failsafe and strong enough to ensure that if the primary is accidentally engaged or fails, the hood will not open and blind the driver while the vehicle is in use. It can be opened from outside the vehicle after the primary is disengaged.

Latch developer:  I'll build two latches of strong material. It should be enough.

Test engineer:  I will test the prototype under the most extreme, windy conditions I can think of. Based on how the latches were engineered, I think it is most likely to fail if jarred, if there is sufficient wind putting stress on the latches, or the owner may believe the hood is closed when it actually is not completely closed.  I will use wind tunnel, off road simulations, and hood drop tests to simulate primary and secondary latch failures.

Product Manager, User Acceptance Testing:  As the business stakeholder, I'll make sure that the product does what is expected, enough that the product solves the business need (eg resulting in purchase).

However without the test engineer in the picture this becomes a blame game when customers thought they closed the hood but didn't close it all the way, especially if the average reasonable person thinks the hood "appears to be" closed.

This degrades into a blame game.  The latch developer is blamed for creating a latch that may not always close properly.  The product manager is blamed for not specifying how the customer would operate the latch, and says the developer should have picked a design that would account for something obvious like being able to easily operate the latch. And so the next time the product manager tries to be much more specific, defining not the problem to be solved but how the solution should be built, restricting the developer from choosing a better solution, and things just get worse from there.  Product managers writing functional stress tests instead of focusing on user acceptance, "will the customer buy/use it" tests.  Developers prioritizing scope over product quality. No one takes ownership of the quality, so the quality degrades without the proper expertise to examine and test the solution in the context of how this specific solution will really be used.

A simple part becomes a recall, costing millions of dollars to repair.

I'm not saying that's what happened for the car recently mentioned in the news, but if you followed my example, you'll realize that the product manager and the latch developer both did their job. The product manager defined the problem to be solved, and the developer created a solution. 

Without someone responsible for quality control, no one will realize there is a problem with the solution that was chosen. The product manager isn't technical enough to know how this implementation could fail, and the latch developer isn't aware that when the latch is attached to a hood, the "hood with latch" might not work as intended.

So yes, quality assurance engineers and test engineers, we need you. We might take you for granted at times, but bad things happen without you - that's when we realize your value.

Friday, February 19, 2016

Phones can't multitask

More megahertz, bigger batteries, better screen, ok that's nice but it doesn't really give me the features I need!

The smartphone industry (hardware and software) have made huge technological leaps over the past 5 years in terms of battery solutions, screen quality, and the sheer usefulness of the apps. I can now go anywhere and get the local news, find my lost keys, check for food allergies on any product with a bar code, get notified about events that might interest me the moment they're scheduled, track the location of my cat or kid, listen to music, allow a cashier to laser scan a bar code on my phone, and get notified about a kidnapped child in my area,... so long as I don't want to do all of those things at the same time!

Current phones have amazing capabilities but also severe restrictions for multitasking.  Most people either wipe their phone out periodically and start over or but a new phone (which effectively doors the same thing). This is what could be improved to make multitasking better.

RAM:  everybody has an app.  Whether you're searching for your next home, a deal at your favorite restaurant, or your missing wallet, our phones connect us to the whole world through apps, but if you have more than about 8 apps looking for information on your behalf your phone isn't able to run all of those programs at once just because of memory restrictions. You'll phone will suck your battery dry trying to juggle them in and out of memory. What do the phone specs give you when you look for that next gigabyte of RAM?  Storage space. "Memory" is storage space according to the major cell phone providers. If I just had more RAM, I wouldn't need an expensive battery or external emergency battery.

Cores:  cores are basically how many things can your phone do at the same time. I think most smartphones are quad core, so the operating system schedules 4 items to be processed at any given instant, giving the illusion that the phone can do hundreds things at once by juggling them around, but really it's just 4. If the OS didn't have to juggle the RAM quite so much it would free up one of those cores, but even then, with all the apps we want to run at once there needs to be either more cores, or more efficient scheduling to run them all. Most apps are just monitoring data feeds for something new to notify us about, but seem to be idle until an event happens. No, they're not idle, just quietly checking for data to send your way.

Antennas:  wifi, GPS, telephone service, mobile data, and Bluetooth - all of these things use an antenna. Most of these are shared single connections that the phone manages, but not Bluetooth. I have a Bluetooth headset, a Bluetooth crash monitor in my car that will call for help in an emergency, and about 6 TrackR Bluetooth devices that monitor the location of my things so that I don't accidentally leave my keys, wallet, etc. behind (I would connect 10 if I could).  If my phone successfully maintains all of the Bluetooth connections, I lose GPS. With so many connections, it's no surprise that I have connectivity problems, either because the phone has insufficient resources to manage everything that's happening (see RAM, cores above) or because there aren't enough antennae to manage it all, but that's a guess.  At the very least someone needs to come up with a better way to manage connections to local gadgets, either an improvement to Bluetooth or a replacement for it akin to TCP/IP on wifi.

What's planned?  Well, I just checked the rumors for the Samsung Galaxy S7 expected to release this Sunday to see if any of these things will be addressed, sounds like more screen improvements. :-/  not a word about the stuff I care about.

I can't be the only one. Is anyone else running into this?