Numeric defect related goals. Have you experienced the syndrome ?
Have you ever heard a team owner (read Leader, Manager or whatever) of a Testing team say any of the below-
- My study of Defect Prediction model suggests that this project should have 400 bugs. Why have we found only 50% so far ?
- You have logged less number of bugs in this cycle. Please pull up your socks.
- Your performance is not upto the mark. Please log more bugs.
- Your goal is to log 50 bugs in this cycle.
- In this quarter, you need to improve your bug logging rate.
If you haven’t heard any of the above in your Testing experience, I would consider you lucky, you have been in a good company. Given the professional situation we stay in, i am convinced that many of the test engineers might had to go through the situation where in they were assessed or recognized solely based on the number of bugs they report, no matter what that means.
Is it worthwhile having Numeric defect related goals ?
It usually makes me wonder about the benefits of having the numeric goals bestowed upon the Test engineers. I really fail to find many.
One may argue that having a numeric goal as a reference always keeps the test engineers focused on the goals that they need to achieve. This may be fair for some but this explanation does not capture the overall scheme of things. Consider the below quote by Ichek Adizes that i recently read-
Managing only for profit is like playing tennis with your eye on scoreboard and not on the ball.
To me this quote clarifies a lot of things. With due credit to the author of the quote, i would take the liberty of tweaking this quote a bit for our profession. Read this-
Managing only for number of bugs is like testing the software with your focus on a mere number to achieve than for the love of profession.
Doesn’t the above quote (or requote i say) brings out the essence of the ill effects a senseless numeric goals has on the most important assets of a testing group- The people. The people who are passionate about Software testing would resonate that what interests them towards this craft is the process of learning something new about the Software, having or working to build a unique perspective to assess the software, to solve the problems of complex nature, to apply new techniques, to learn and relearn and many many more things. And the end result of application of this passion, interest, skill is the bugs that we see testers find.
Having focus only on finding a certain number of bugs seem more like robbing a Test engineer of all the greatest perks of work that result in finding the good bugs.
The need for Positive motivation to find bugs:
Before I sound like going unidirectional, I do recognize there is a need of certain motivation that can possibly help find good defects. And certain people do apply the thought of keeping numeric goals to improve defect rate and probably may achieve certain success but this certainly leads to defocusing of Test engineer's energies to a mere number leaving him/her not enjoying the overall process of testing, which is certainly not good for the overall sake of the profession.
I do believe that in addition to educating Test engineers on the right knowledge and skills, as a Test group owner one needs to have effective motivational strategies in place as well. How-so-ever tempting the business of numeric goals may sound, somehow it doesn’t seem to convince me fully. I would certainly worry if any project in question doesn’t have any bugs found but then such a situation is ideal for further analysis and to be figured out the root cause of such an issue rather than just setting a number goal. This may work sometimes (may be if you are dealing with staff who isn’t passionate enough about Software testing and treats it as just as a Stop gap arrangement) but not in majority of situations.
Try using Placebos for silent motivation:
In his book, Stop the Excuses , Dr. Wayne W. Dyer says-
That mind controls the body is hardly up for dispute. You have probably heard of documented studies where sugar pills given to control group believing that they're a remedy for, say, arthritis, turn out to be as effective as the drug being administered, for arthritis. This placebo effect apparently occurs due to the belief in the effectiveness of the pill.
Placebos are quite widely used in the field of medicine. It’s a kind of make believe treatment in which patient believes that he is being treated upon with medicine for some ailment but in reality he is just being sugar pills. This makes his mind believe that he is being treated rightly and to some surprise the medical field has found considerable success with such a Pseudo treatment.
Placebos and Software testing- Are they related somehow ? Some might think this reading the above. But i feel there is an inherent connection there. Consider a situation that you are dealing with a complacent team who believes that they have tested all that could be possibly tested in a product and there are "no" bugs in the product (In my experience, I find teams feeling confident, may be over confident to make such a tall claim, which is improbable). In these situations, one could either believe what the team says or may be "push" them to find more bugs by setting numeric goals. But I do feel Placebos do work quite well there. Just have the team swallow a Placebo that "Development team feels that there is some part of code which is vulnerable to bugs, many bugs." I call this Placebo because as this is no less that a sugar pill that helps Testing team deal with Complacency and act as a certain source of motivation. Complacent test engineers, no matter how much skilled they are- tend to let go of opportunities to find good bugs. You might even be surprised to see the number of test ideas that starts flowing out of the minds that weren’t even considered before. All this just by chewing (not literally!) an artificial Placebo.
The overall point I was trying make is that there are good ways, better ways than mindlessly directing numeric defect related goals.
Do you tend to agree or disagree ? I would certainly love to hear!