I ship the Information!

I was following a discussion thread in Software Testing Club. When I read the title of the discussion "No issues in application" with description "What do you do when you are not finding any issues in the application you are testing?" I did not know what author had in his thought while he wrote the description. Was he witnessing, what he had written in title and description or it was a thought asking what should be done when (it is or it will be or it was) witnessed? This question remained along with many other views of me for the very same title and description.

Following the discussion, I understood that the author was witnessing what he had written in the title and description of the discussion thread. The reason for witnessing this might be the illusion, biased and the other things too if it exists and one discovers it. Focus and defocus might help here to discover, generate, invent and explore the new ideas -- the heuristics.

The few words, phrases and sentences from the discussion helped me to unlearn and learn -- how can I infer and perceive the information:

/* These are the few words taken as it is from the discussion thread. */

• wrong tests.
• if the main functional user test cases pass, then the quality of the sw may be good enough (depending on how critical the sw be in the real world).
• Why are there no defects in this application?
• Pesticide Paradox.
• there are always defects, they must be somewhere.
• Perhap I could see where other defects have been reported in the past and test around those areas.
• Why are the test cases all passing?
  
• Have I tried end-to-end or process driven testing?
• Ship the application.
• I typically step back and look at the tests I've run and try to identify patterns in what I've been doing. With my next set of tests, I attempt to violate those patterns.
• Julian Harty's techniques around six thinking hats for software testers.
• trying different tours of the application to generate new test ideas.
• better develop my mental model of the application.
• pairing with others on the project team.
• are you asleep and dreaming this.
• It really meant that the test cases you are running - either they are not updated ones or just clicking on "PASS" on any behavior.
• This scenario will not come.......if it comes well.....time for all the testers to pack their bags. :)
• Take a step back and think just what you are supposed to be doing and what you are actually doing.
• If you're not finding any defects then there's something fundamentally wrong with your approach.
• if you're not finding problems then you're doing something wrong.
• In all my 20 years of testing it have never happened!
• If am not finding any issues in application as per System Requirements / use cases. i start doing negative testing.
• try to brake Application by any means. if it still survive then the application is ready to be Delivered.. :)
• Application has gone through all the negative testing and has survived, however I still have couple to days time to test.
  
• Hence I was trying to find if there are any issues.

/* End of selected words from the discussion thread. */


Looks like the application that was said, might have been shipped which made to close the discussion thread.

When I began to learn and unlearn my understandings with the above words, phrases and sentences which were written in the discussion thread, the skeptical notions I am inferring and perceiving are as below:

• Test(s) give the information when they are executed. I don't know and I am investigating yet how to classify, call and negotiate them as wrong tests and correct tests. Words are always ambiguous. In this context, both wrong test(s) and right test(s) provide the information.
  
• Testing provides the information. If my test cases fail, still I am getting the information which might be of value to the people who matter. If quality is value to the people, then isn't the failed information is useful intellectual cognition? Why test cases marked as passed the test(s) alone get the importance?
  

I understand 'Quality' is not in the software which is used, but it is the emotion of the user when the model is used. I remember the words of Gerald M. Weinberg "Quality is value to some person". If I wear a silk (which is said to be of high quality by fabric person who weaved or manufactured that silk garment) shirt of bigger size (than I need) that hides my trouser when that shirt is not tuck in or which makes me not to wear trouser or I am unable to tuck in the shirt due to its size -- am I getting quality on wearing it? Am I satisfied, enjoying and loving to wear that silk shirt?

It depends from a user to user along with other factors which can have the influences for wearing a silk shirt or not. And if I don't like to have that shirt, where is the quality for me in that shirt which might have passed the test, to say the silk used in fabric is of high quality? But, if I wish not to spend money on buying a trouser and this bigger size silk shirt covers my shoulder to ankle and I am comfort with it and happy using it, might be the value for the money what I have given is with me.

• When idea(s) or heuristic(s) does not bring the information that tells it fails to meet the claims, it is not a failure idea. It is the information that shows how the model does what is supposed to do, in yet another way too. If the idea(s) or heuristic(s) does discover the information, how the model fails yet in another way of using it -- it is not a success. It reveals another way how the model does not do what it is supposed to. One can get such plenty of information which makes him or her to infer the model (appear as) to work or not work. So failure and success cannot be defined with the information given by the Tester's Testing. And, may be the model cannot be shipped by decision makers, with tester(s) information alone.
  
• I am not against test case writing. The first law of documentation is to document it (from Rapid Software Testing). If I am just executing the steps in test cases, it is more of like verification and validation of test cases and not the testing of model under test and testing of the being executed test cases. Writing the test cases is also an art as testing is, where both require skills, practices, failures and the explorations.
  
• If System Requirements are written in a document, then is that document itself is the requirement(s)? Requirement document(s) has (ambiguous) words which describe the requirement(s); and the document(s) are not the requirement(s). Man makes mistakes while writing, so computer can also do the same.

Computer or document model does not tell the requirement(s) or whether what I am writing is correct or incorrect or this is what was expected by the user or user does not want this. In simple, document or document software model does not ask questions or answers for what is in my thoughts about requirement(s). It just processes whatever I give, if it is able to take what I give and I perceive this as computer as understood my question and given the solution; where all these are nothing but an illusion. It appears to me as document software model has done what it was desired to do.

If sensing a impulsive feel like model under test is matching to requirement, then it is not too late to understand, testing is not matching or not just matching alone (as matching may involve few tests). 'Matching' reminds me the writings of Michael Bolton on Checking, Testing and Problem. One may feel the information from matching are sufficient but does not feel information from testing are enough, since it always reveals or shows new things apart from those seemed as matched and did not match.

• Negative Testing and Positive Testing might be the textbook words. Whatever it might be, the information from so said 'Negative Testing' or 'Positive Testing' adds its own values. If inferred as value given from so said 'Negative Testing' is high than so said 'Positive Testing', then one day it might happen such that positive(s) can be negative(s) and negative(s) can be positive(s). I remember the words of James Bach, "Testing is infinite process of comparing the invisible to the ambiguous so as to avoid the unthinkable happening to the anonymous."
  
• The information obtained from testing is more than the ambiguous strings 'PASS' or 'FAIL' or 'NOT EXECUTED'.
• Tester cannot take the decision or tell to ship the model under test, but might make the influence on the shipping decision by decision makers with the information given from her or his testing. I don't know yet, how the moment would be if the decision maker is the only tester.
• Challenging the assumptions might challenge statements, "No issues in the application" and "What do you do when you are not finding any issues in the application you are testing?"
• Saying "I don't see any problem here" is a problem which might produce 'n' number of problems. Still seeing anything as 'no problem here', then questioning 'when it can be a problem?' or 'how it can turn to be a problem?' might help.
  

I owe all the credits to the people who participated in the said discussion thread.

A Tester does not ship the model he or she is (or will be) testing. A Tester ships the information from her or his testing.

First test consultation starts with heuristic -- Kulja Sim Sim!

My friend brought me an opportunity to meet his friend, who wanted to test his programming instructions before presenting it to the exam and viva panel. It was a gift to me by these friends on 15th August 2009, to also experience testing in new way which I was not witnessed so far.

On reaching friends house, he gave a glass which had hot coffee in it. When I began to drink, my tongue and lips felt the heat of the coffee. My friend's mother said, "Opening the lips to drink hot coffee will make you to feel the food what you eat tasteless for few days" -- this heuristic gave me a test idea.

I was said about what I will be testing. Before going further, I confirmed my mission by asking what you are looking from my testing and what kind of information was expected. Further gathered details about what was available and what was not available as per the friend who gave me the mission to test. I found useful and vital information while we were discussing and I kept it writing down. And finally I was given the model to test.

Began to use the model and learn how I can make use it for my purpose. On exploring found the information where the model crashed if the user entered incorrect password after 6 times consecutively, though it locks the account of the user on entering the incorrect password for first 3 consecutive tries. It needed to bring up the instance of model again to continue using it.

Continuing my explorations, friend beside me started to have the coffee for second time. He said "it is much hotter than the previous one and my mouth opens for this hot coffee when I say filter coffee". These words made me to remember the words -- "Kulja Sim Sim" said in tale of Ali Baba and Forty Thieves. And the words that are said by Ali Baba to open the doors of the cave in the tale Ali Baba and Forty Thieves, became the heuristic for me to test for hidden treasure of model over the network.

Began to find the behavior (how the model opens to network when user starts using the model) how the user credentials was transferred to the file server on same machine. Observed and inferred that any hacker could find user credentials without much trouble though it appears to be the cipher text. This can be one of the critical information in the software model that costs the user for using it. Exploration went on for around two hours.

I gave information that was gathered, to my friend -- who gave me test mission and model to test for the test session. We analyzed each observation made; and found much more information without using the model also. We looked into the API that was used to transfer the user credentials and found that it was primitive one.

My mission maker for this testing said, these information helped him to present demo to exam panel and might find information while demo also. What he said was true -- that is we gather information during demo but demo cannot be testing in one view. If the demo says only what works and tells the false information about the model being demonstrated then it cannot be testing (my understanding as testing means saying truth what it is).

But in other view demo brings information during and after using model also, if person giving the demo is attentive during demo with the behavior of the model for his or her action, response of people in the demo etc. The point we discussed here was for what level we were involved in demo and what kind of attention we had for testing during demo also. Probably not fully involved in testing and this might not make full potential use of brain to think about contexts to tell -- it does not does for what it is assumed that it does.

I will know what works and what does not works at least with the information I discovered. If some other person gives the demo of the model who does not know what works and what does not works, but knows what model should do -- can turn to become one more informative session (or not) for (all or few or no) people in the demo. This comes with the cost for showing the model that does not do what it was wants to do as desired and value of knowing information that it does not do what it was wanted to do. But need to analyze which weighs more i.e., cost or value or both.

I shared these thoughts to my friends. Thanks for both of you for giving me an opportunity to see new flying wings of testing and learning.

I am h(f)ired because I use my brain!

I was moving to house previous day. It was late night and did not find any public transport for 25 minutes. I was talking to one my colleague who works with me by discussing and debating. Finally, we found a bus and moved inside the bus.

My friend found his seat in the front seat of my seat, where I found a seat for me. Beside me a person was seated and was looking seriously looking at me. We began our conversation for 30 minutes without knowing each other name. I would like to call him as a Fellow Traveler --- he gave me good lessons to learn and I learned.

Me: Hi!
Fellow Traveler: Hi! Seems like you are looking for a job.

Me: Yes. I always look for that as it always helps me to know where I am. Seems like your employer is ABC.
Fellow Traveler: Not now. Today I work for JKL.

Me: Any opening for learners in your workplace.
Fellow Traveler: Learners?

Me: Yes.
Fellow Traveler: What work you do in your office?

Me: I am a Tester.
Fellow Traveler: What tools do you use for testing?
Me: I use my brain?
Fellow Traveler: What? Brain? I asked tools.
Me: Yes, that is what I use and that tool works for me.

Fellow Traveler: Are you not fired yet?
Me: I was hired as they discovered that I used my brain. Don't know when they will fire me, if I don't use my brain or for using brain.
Fellow Traveler: Looks funny! What they say to you when you use your brain?
Me: "When you are going on vacation?"

Fellow Traveler: Without test tools, how will you do that? To come in to my work place, you have to know testing tools.
Me: What does the 'testing tools' means, which you are talking about? What does 'you have to know testing tools' means?
Fellow Traveler: Tools that is used to automate the testing which you do.
Me: I test almost what I don't see and see, I don't hear and hear, I did not know, that I don't understand and I understand by learning from testing it. Where shall I place your "automate the testing which you do"?

Fellow Traveler: Oh! You are cracking my knuckles man. I meant what test automation tools you use for the testing you do in your work place.
Me: I know to use them and learning to use them by knowing them i.e., the test automation tools you just said.

Fellow Traveler: Do you know WinRunner?
Me: I can run the Winrunner.

Fellow Traveler: Do you know QTP?
Me: I do Questioning Tests Passionately.
Fellow Traveler: I never heard that. Do you know any other abbreviation for that?
Me: I can keep telling for that. The other one now that I have for you is...
Fellow Traveler: I am wondering, why you are not fired yet? Have you used LoadRunner?
Me: I make loads of tests to find information.
Fellow Traveler: What that loads of tests you do?
Me: I discover, invent, explore and reinvent the questions. This helps me to find the information and some time it does not; but provides the other information, which helps me again to create the loads of tests I said.

Fellow Traveler: Do you know Rational Robot?
Me: I am a human, who is learning to think rationally.

Fellow Traveler: Common man, don't poke with your answers. How they are bearing you?
(my friend in front seat was looking at us and was smiling, listening to our conversations)
Fellow Traveler: To work in my company you have to know these tools?

Me: Which tools?
Fellow Traveler: WinRunner, QTP, LoadRunner, Rational Robot.

Me: Did you skip any more software test automation tools?
Fellow Traveler: Why?
Me: There are also open source test automation tools, which might be handy depending upon your project contexts. I wanted to know, why the tools you said, are used in your company?
Fellow Traveler: We make applications that are huge and used in real time by several people at a given point of time. A tester cannot do the testing for this kind of applications.

Me: OK. Who made the applications you told?
Fellow Traveler: Are you deaf? I said right, we make applications that are huge.
Me: How huge it is? I want to know what the huge means to you.
Fellow Traveler: You are funny man.
Me: Yes it appears so. When your people who are also human, can make applications, which you said, that were huge, why can't the human test it?

Fellow Traveler: What do you want to say?
Me: You said your applications were huge and used as a real time system. I don't know how huge it is and what the huge means for you. I can see every application as real time system and also as non-real time system. Putting the huge application into the hands of automation will be like, for an example, you are sitting in Volvo bus which is air conditioned, quite luxury seats and the other features it has. I have never traveled in Volvo bus.

But, the Volvo bus in which you are seated now, say it is traveling in road traffic of Bengaluru where the person driving the bus is covered his head completely such that he can only breath and survive but not able to see anything neither hear anything but can move his hands and body while driving the bus.

Are you safe in that bus though it is Volvo? Might be you are safe till some extent or to the full extent of your journey in that bus. But what is the guarantee that you are safe? Also, when the person driving the bus in which you are seated is able to see and hear by moving his hands and body, still no guarantee of your safety of your journey. All what we have is some words or claims that say using service(s) of so and so will make us so and so which are either advertised or communicated orally or done by both the way. And we are all factors of it, when we witness it.

But the risk might be bit lower when compared to the other case and all it depends on the people inside the bus, the person driving the bus, and the tool being used -- the Volvo bus. Right? This way, if you model the test by testers hope the risk of using the huge application you said might be lower or not at all; then you need to think about the strategies to work on.


Fellow Traveler: You are saying that test automation is of no use?
Me: I did not tell those word anywhere. But, knowing how to use them for helping testing what I do is required. Test Automation Tools does not do the testing what I do; it might help my testing activities, which is human tasks and to know what extent it helps is useful data. But, it cannot replace the human tasks without the intervention or interpretation of human and human thinking. Test Automation Tools cannot be paralleled with human for the tasks each do on their own cognitive and intuitive.

Fellow Traveler: I am hearing strange things.
Me: Me too sensed the same sometime back, until I failed and now too I fail. Automation tool (i.e., a model) with bug (any software is not free of bugs) automated another model, which also has bugs. Something interesting, when I witnessed that.

Fellow Traveler: OK man, see you again.
Me: I wait for that moment. Bye! What work you do for your current employer?
Fellow Traveler: I work on systems.
Me: Systems? What systems? What does it mean?
Fellow Traveler: I have to leave; this is my stop where I get down.
Me: Bye.


Later my friend in the front seat began our conversation about the talk, which I had with fellow traveler. I learned from those conversations of fellow traveler and my friend. I hope the fellow traveler who was in conversation with me, might see this writing one day and we meet again.

The software (and combination of hardware & software) we make is a kind of an automation of human work, is what I have understood. I was questioning myself, "Is that tools are fallible?" "Only tool does experiences the failure?" And I kept writing many such questions and wrote statements (towards and against) as much as possible for the each questions. I knew human is fallible, then the heuristics identified by human are fallible, then the model built by human which are set of heuristics (programming instructions -- in this context) are fallible. A question remains is "Which is not fallible and not (to) prone for failure(s)?" If any such exist with us -- when (not), how (not), why (not) it is fallible and not (to) prone for failure(s)?