I was following a discussion thread in Software Testing Club. When I read the title of the discussion "No issues in application" with description "What do you do when you are not finding any issues in the application you are testing?" I did not know what author had in his thought while he wrote the description. Was he witnessing, what he had written in title and description or it was a thought asking what should be done when (it is or it will be or it was) witnessed? This question remained along with many other views of me for the very same title and description.
Following the discussion, I understood that the author was witnessing what he had written in the title and description of the discussion thread. The reason for witnessing this might be the illusion, biased and the other things too if it exists and one discovers it. Focus and defocus might help here to discover, generate, invent and explore the new ideas -- the heuristics.
The few words, phrases and sentences from the discussion helped me to unlearn and learn -- how can I infer and perceive the information:
/* These are the few words taken as it is from the discussion thread. */
- wrong tests.
- if the main functional user test cases pass, then the quality of the sw may be good enough (depending on how critical the sw be in the real world).
- Why are there no defects in this application?
- Pesticide Paradox.
- there are always defects, they must be somewhere.
- Perhap I could see where other defects have been reported in the past and test around those areas.
- Why are the test cases all passing?
- Have I tried end-to-end or process driven testing?
- Ship the application.
- I typically step back and look at the tests I've run and try to identify patterns in what I've been doing. With my next set of tests, I attempt to violate those patterns.
- Julian Harty's techniques around six thinking hats for software testers.
- trying different tours of the application to generate new test ideas.
- better develop my mental model of the application.
- pairing with others on the project team.
- are you asleep and dreaming this.
- It really meant that the test cases you are running - either they are not updated ones or just clicking on "PASS" on any behavior.
- This scenario will not come.......if it comes well.....time for all the testers to pack their bags. :)
- Take a step back and think just what you are supposed to be doing and what you are actually doing.
- If you're not finding any defects then there's something fundamentally wrong with your approach.
- if you're not finding problems then you're doing something wrong.
- In all my 20 years of testing it have never happened!
- If am not finding any issues in application as per System Requirements / use cases. i start doing negative testing.
- try to brake Application by any means. if it still survive then the application is ready to be Delivered.. :)
- Application has gone through all the negative testing and has survived, however I still have couple to days time to test.
- Hence I was trying to find if there are any issues.
/* End of selected words from the discussion thread. */
Looks like the application that was said, might have been shipped which made to close the discussion thread.
When I began to learn and unlearn my understandings with the above words, phrases and sentences which were written in the discussion thread, the skeptical notions I am inferring and perceiving are as below:
- Test(s) give the information when they are executed. I don't know and I am investigating yet how to classify, call and negotiate them as wrong tests and correct tests. Words are always ambiguous. In this context, both wrong test(s) and right test(s) provide the information.
- Testing provides the information. If my test cases fail, still I am getting the information which might be of value to the people who matter. If quality is value to the people, then isn't the failed information is useful intellectual cognition? Why test cases marked as passed the test(s) alone get the importance?
I understand 'Quality' is not in the software which is used, but it is the emotion of the user when the model is used. I remember the words of
Gerald M. Weinberg "Quality is value to some person". If I wear a silk (which is said to be of high quality by fabric person who weaved or manufactured that silk garment) shirt of bigger size (than I need) that hides my trouser when that shirt is not tuck in or which makes me not to wear trouser or I am unable to tuck in the shirt due to its size -- am I getting quality on wearing it? Am I satisfied, enjoying and loving to wear that silk shirt?
It depends from a user to user along with other factors which can have the influences for wearing a silk shirt or not. And if I don't like to have that shirt, where is the quality for me in that shirt which might have passed the test, to say the silk used in fabric is of high quality? But, if I wish not to spend money on buying a trouser and this bigger size silk shirt covers my shoulder to ankle and I am comfort with it and happy using it, might be the value for the money what I have given is with me.
- When idea(s) or heuristic(s) does not bring the information that tells it fails to meet the claims, it is not a failure idea. It is the information that shows how the model does what is supposed to do, in yet another way too. If the idea(s) or heuristic(s) does discover the information, how the model fails yet in another way of using it -- it is not a success. It reveals another way how the model does not do what it is supposed to. One can get such plenty of information which makes him or her to infer the model (appear as) to work or not work. So failure and success cannot be defined with the information given by the Tester's Testing. And, may be the model cannot be shipped by decision makers, with tester(s) information alone.
- I am not against test case writing. The first law of documentation is to document it (from Rapid Software Testing). If I am just executing the steps in test cases, it is more of like verification and validation of test cases and not the testing of model under test and testing of the being executed test cases. Writing the test cases is also an art as testing is, where both require skills, practices, failures and the explorations.
- If System Requirements are written in a document, then is that document itself is the requirement(s)? Requirement document(s) has (ambiguous) words which describe the requirement(s); and the document(s) are not the requirement(s). Man makes mistakes while writing, so computer can also do the same.
Computer or document model does not tell the requirement(s) or whether what I am writing is correct or incorrect or this is what was expected by the user or user does not want this. In simple, document or document software model does not ask questions or answers for what is in my thoughts about requirement(s). It just processes whatever I give, if it is able to take what I give and I perceive this as computer as understood my question and given the solution; where all these are nothing but an illusion. It appears to me as document software model has done what it was desired to do.
If sensing a impulsive feel like model under test is matching to requirement, then it is not too late to understand,
testing is not matching or not just matching alone (as matching may involve few tests).
'Matching' reminds me the writings of
Michael Bolton on
Checking,
Testing and
Problem. One may feel the information from matching are sufficient but does not feel information from testing are enough, since it always reveals or shows new things apart from those seemed as matched and did not match.
- Negative Testing and Positive Testing might be the textbook words. Whatever it might be, the information from so said 'Negative Testing' or 'Positive Testing' adds its own values. If inferred as value given from so said 'Negative Testing' is high than so said 'Positive Testing', then one day it might happen such that positive(s) can be negative(s) and negative(s) can be positive(s). I remember the words of James Bach, "Testing is infinite process of comparing the invisible to the ambiguous so as to avoid the unthinkable happening to the anonymous."
- The information obtained from testing is more than the ambiguous strings 'PASS' or 'FAIL' or 'NOT EXECUTED'.
- Tester cannot take the decision or tell to ship the model under test, but might make the influence on the shipping decision by decision makers with the information given from her or his testing. I don't know yet, how the moment would be if the decision maker is the only tester.
- Challenging the assumptions might challenge statements, "No issues in the application" and "What do you do when you are not finding any issues in the application you are testing?"
- Saying "I don't see any problem here" is a problem which might produce 'n' number of problems. Still seeing anything as 'no problem here', then questioning 'when it can be a problem?' or 'how it can turn to be a problem?' might help.
I owe all the credits to the people who participated in the said discussion thread.
A Tester does not ship the model he or she is (or will be) testing. A Tester ships the information from her or his testing.
