API Testing -- Did you come across these questions?

Do you test API? What tools you use to test API? Hey, you also test the REST APIs? Automate and run the suite that is sufficient to APIs, isn't it? Have you asked any of these questions? Other person have asked you any of these questions? Or, you heard such conversations?

I will share my learning what I'm making by testing the APIs. Use the label "APIs"in this blog for finding related posts to it.

When I hear above said question, the first question that I ask is, "What do you want to accomplish and know by testing the APIs which you are talking about?".  I see no response coming back most times. Yes, we all use APIs! And, we will continue to use! What is that I want to know from testing the APIs?  How do I test? Why at all I have to test for APIs?

I got notification on my mobile phone from the app I have installed?  The APIs helped to receive that notification.  Did I see the stock market's number updating on my desktop or mobile app? That was with the help of APIs. Do I see anyone monitoring the health and fitness with the wearable device and then transferring that data to mobile or desktop? That happened with help of the API.

In the next post, I will be writing the below mentioned.

• What is an API ?
• The types of API
• REST architectural design for the Web

IME Test Model - RICH VIP MUST PLUG AND HE PUTS LOCK

I was approached by my fellow Software Testing practitioners - Shristy and Suchismita for knowing and to have better structure for testing the IME - Input Method Editor.  On listening to their context of current practice and what they wanted to know by testing, I learned for first they need the essential design components of today's IME.

I had to make sure that, this learning is fair enough to start and from here they can assist themselves. On brainstorming together for few minutes, we learned, it is good for a start to have the key integral design components of IME. Have this, the testing can be channeled well in those areas as and how the context demands on priority.

Now we had listed down the IME design components fairly sufficing to start. What are the tests to be done on the app under the IME design components? It depends on the context of testing.  The testers here were able to identify the tests based on the context needs. But the challenge for them was -- knowing how to approach and categorize the IME app.  Now it is addressed with this IME Test Model, tester can help quickly to visualize and pick up the tests under the hood of respective design component of IME.

Credits are for Shristy and Suchismita for pairing up with me and in framing the mnemonic of this Test Model and categorization of it.  This can be referred and used for Android IME and iOS IME apps testing.

A Test Design being guided by the Test Strategy

In this post, this image looks to have too much structures around it.  But, it is not. It is very simple one as I see it.  It is simple when I happen to understand the Test Strategy and what is the help out of it.  Having said this, why is that simple, because, it is from the fundamentals and nothing else.

If I understand, Test Strategy is set of ideas which directs my test ideas and test execution, then the test strategy has influence on Test Design.  Where the test design consists of these elements necessarily within it and it is as below.

1. Test Model
2. Knowing the Oracle
3. Knowing what is my Test Coverage
4. How do I operate my tests and execution of it.

The below picture says the same in the equating form.

The same applies when I have to figure out how to approach the automation for assisting my testing. I use this and when it requires a change in the context, I will do it accordingly and try to get most out of it.

Update (on 23rd Aug 2016):  I unlearn and learn the equation represented as this is more meaningful:

Test Strategy = { Ideas <--> Test Design + Test Execution}

As the test design and test execution in turn helps the ideas to evolve better as the testing progresses.

Inside Out of Test Design: Test Technique Identification with help of Test Model and Strategy -- Part 2

From the overlapping, I move ahead to learn how the test, technique and design is part of each other, I see, the three more now by adding to conversation.  It is -- Coverage, Action, Information (Potential Problem), and Assessment. This looks to be me as in the below image. 

Figure: Test Design and Identification of Test Technique

I understand this for Test Model -- a representation by which I'm learning the product either associatively or relatively or by both means. Further, I can still refine the model to specific learning i.e. to assist in obtaining the specific type of information from my testing.

Moving ahead, I will learn what information is expected from my testing. Knowing this, I will have to build a Test Model or use a existing one, to understand the product and my tests which I will be executing. The Test Model will help me to learn critical factors associated to product, environment, project and related components.

Now, I will have to choose the test techniques or build one. This has to assist me in building and strengthening tests by involving the heuristic within it also by identifying the heuristic from Test Model and tests.  This heuristic will help me to understand the outcome of test.

When I have figured out (or as I figuring out) the Test Model, Test Technique, and Heuristic for the test, I will have know what is the Test Coverage to be accomplished in context of testing.  This gives the dimensions and influencing description to say why is this my Test Strategy, and why I have made use of this model, technique, and heuristic to arrive at this coverage. As I execute my test and the way in which I record it, this overall constitutes the part of my Test Design.

Looking at the techniques, I have known there classification from the Software Testing books as Black Box, White Box and Grey Box. I'm learning there is much more within each of these types than the books say.

To summarize my so far learning, I see, the Test Techniques is one of the outcome of Test Strategy. The test strategy directs me to pick or build a Test Model. With the help of Test Model(s), I will execute the test by learning how I have to execute it and to what extent using the technique(s) (which is also a heuristic).

Inside Out of Test Design: When each disguises each other - Test, Design, and Techniques -- Part 1

I have been and continuing to be mesmerized about the Design Patterns in programming space. While I'm continuing to look at the Design Pattern, I ask myself is there anything similar in the 'software testing' space? If software programming and software testing are subset of a super set, the engineering, then design pattern and test design are to each other. That is, the both exist to support each other well enough in the context. Isn't it?  This is the question among lot other which runs in me when I think about Test Design.

Further, I go into my thoughts (of today) for letting know myself what I understand for 'test', 'design' and 'techniques', I hear below from me.

• test -- An experiment which is an open ended and whose outcome is unknown.
• design --  The set of ideas which says how to (I) use it or how to go about; and how I see it further to use it by learning. How simple it is to use for me, those are the attributes of design with the learnability in context being prominent.  In other ways, it comprises of pattern as soon as I say, 'design'.  Each design has a pattern isn't it?
• technique --  applying; i.e. a skill being applied to study, observe and evaluate a subject of interest further.

While I'm learning this, do I see a familiarity or the familiarities in these three above said?  May be this familiarity or these familiarities, it disguises each other while they are distinct and co-exist mutually in practicality of Software Testing practice.

Why one cannot see the differences or draw the differences between these other is, a simple fact. Yes, the fact, which is not studied quite often; hence it is a fact.  The unquestioned fact is, all three of these are heuristics - which helps in discovering, learning and solving, while it can fail as well.

A test does not help one to learn, discover and solve? But how a test can fail? Test never fails, it provides the information and I tag the label of true positive or false positive to the outcome of it. Likewise, is the design and technique, it helps in discovering, learning and solving; and as well it can fail too. But how do I see the 'fail' in case of design and technique is -- it did not meet the purpose or serve the purpose. Because out of the design and technique, I expect the discrete value which is of usabale, reusable and valuable. Whereas for a test, I see the information is an outcome from the execution of it (or out of not execution as well; not testing is as well a useful test in a context).  This is where the disguising appears by each others to each other.

Do I want "the bug" or "the testing" ?

When a fellow tester or colleague or stakeholder ask, "Let us find the bugs by testing" and "Let us learn the problems and risks by testing", I think the below.

Why testing is often seen or taken granted just to see the bugs out there or the risks residing within, if at all? Do programming is to introduce the bugs and fix them, all in all?  Then why testing and testers are seen as behind the bug always or put around the bugs?

I see this attitude (mindset) and expression of looking and practicing the Software Testing, yielding to experience just the one small part of a petal of the craft. While there are other petals and benefits of the craft, it goes unused and unexplored.  Unlocking from this expression and attitude becomes so complicated, it looks inseparable from the above said thought. The reality is, it is distinct and separate; but, taken as one.

While programming is to solve the problem, isn't the testing is to learn the problem if any in hand. Before this, I ask myself, why there should be a problem always to solve? I see this notion i.e. seeing a problem in first hand, makes the mindset just to focus on the problems rather than placing the problem as one part of the study. That means, problems get associated with everything, then! Why so? We look for it placing the importance for it? Does that importance is studied well to make it important to emerge as a whole than being a part?

To recollect the definition what I remember from Dr. Cem Kaner is, Software Testing is an empirical technical investigation activity conducted to provide (learn) quality related information of product to the stakeholder so that they make informed decision.

Here he does not say about the 'bug' or 'problem' or 'risk' or 'defect'. I see this is so close to daily work I do in my practice.  After this I get the question for myself, "He says, 'information' about quality criteria. Is there anything that is 'not a quality criteria' and still the tester provide the information from her or his testing about that too?"

Leading from here, my thought spans to a branch. This branch gives the thought of the word 'value'. In programming, declaring a variable and assigning it a value is simple? And this assignment takes it's types and namespace in the defined environment.  If this is simple to declare and assign a value, why this value of a product tend to become so complicated when it is tried to in terms of a software product to use?

Now should I learn the value of each and every aspect that is getting collected, then written as part programming and coming out as a usable commodity and the outcome of using it?  Is that value here is -- a bug or problem or risk, I learn from my testing?

Later this points me to line -- "A bug is that which threatens the value of a product". If this is so evident, then, why I hear or asked to look for bugs alone around the product? And, not to learn the value and what is that trying to diminish it in the context?

This further makes me to ask myself, fundamentally, are there anything else in the existence, which is non-bug that can threaten the value of a product and annoy the user? Something as this, programming is just to write buggy code? If so, testing is to look and anticipate just the bugs?

Unfortunately this has crawled so deep in the industry, we want bugs but not the testing. We ask the bugs and not the testing. The impact of this is recursive and passes on to the generation of testers as it is come to my time as well. It makes me to look for bugs (information) i.e. one of the outcome from testing while ignoring and tending to being away from the testing. 

I'm not saying to unroot this or to do anything that helps. But, is it possible to give a thought from the practicing tester before asking anyone to give the thought about this. I want to give my thought about this each time I test to learn what I'm doing and practicing.

That's the common thing to happen! What is next?

The words "hang' and "crash" looks so attention grabbing!  I see these words grabs attention today as well but it does not retain the attention for longer time unless the someone is behind the programmer and tester for it seriously.

Gradually, I'm learning, the crash and hang are something very usual in software applications or hardware products.  Crash and hang are not cruel but it is an indicator of what is being missed to see. Like fever in human body is not a threat; but, it is an outcome of something that is not good in body and an indication.

I have to admit this honestly. I was so thrilled some years back when I happen to see the hang and crash. I use to say it hanged and people use to run for lab.  Unfortunately I was not in a state of mind to tell "that is okay, but what we have to looks is what made it hang or crash".   The hang or crash behavior is an outcome of an actual problem which was never learned by me, then. 

While I happened to practice the test investigation, I learn, hang is a symptom. I fetching as much as information from symptom to unlock the problem node, is the essential part of what I'm practicing now.

Today when fellow practicing testers come saying, "it got crashed", "it is hanged and unable to proceed" and as this, I say "It is a common thing to happen in software products.  Than writing this as a bug, go figure out what is making the product to fall into this state.  If you are not aware to learn that from test investigation, let us pair up and learn it."

Most times or all most every time, the symptom is reported as bug while the problem source is unattended by the engineering teams. Fixing symptom is easier so that symptom is not visible again. But the problem resides and it shows up something else as symptom and one more bug if identified this symptom.

Symptoms are trace backs to the problem source. Not stopping here is a good little small step for being better in test investigation skills.

I Do Not See “I don’t see”

I stopped writing the thoughts which I’m penning now, on listening to fellow testers at desk. Listening to them, I’m making note of the words which they are using in their discussion.  Altogether, I’m out from the thoughts of what I was writing and I’m into the thoughts of what the fellow testers are discussing.  I was writing an incident of code review by a tester (by illustrating it with the snippet of the code) and how the tester figured out the case which automation would have asserted as no problem while it is a problem.

Okay, now I hear the tester saying “not a problem while it is a problem i.e. a bug”.  Hearing this, I say myself, “yes could be the bug in the written automation code”.  But for the testers, who happened to agree mutually later in discussion between each others, they came to consent - it is a bug in the product as well.  While this is happening, I see a tester walking in to the bay and joined discussion by saying, “I read, not everything is a bug what I think as a bug.” Well, now I’m out of my thoughts completely into which I was and I growing more curious to listen the fellow testers discussion.

I asked the fellow testers, "What is a bug then? How you infer about it?" I hear them saying, "A bug is a characteristic which annoys the user" and "A bug is a characteristic or happening which threatens the value of product and product business".  The other tester iterated, "not everything is a bug while I think it is a bug."

Now I have changed my writing thoughts from the automation case study topic to this topic and I started to make note of the words as we discussed.  In discussion, I see a thick line of contradiction implicitly in these two lines – “a bug is a characteristic which annoys the user” and “not everything is a bug while think it is a bug”.

A user perceives the experience is annoying to her or him and take it as a bug while using the product. But the product not taking it as a bug, because, it does not see a threat to the value offered from it. Then what is the bug here?  Should I stand with a user perception of experience or the product’s philosophy saying no threat to value offered by me?

Alongside the thick of contradiction here, this is a thin line of thought differentiating between the two statements – a user experiencing annoyance and a product’s value what it intends to give. In other perception, it is, “whose words matter to label it as a bug and having it in the fix list?”

If I use a product and I’m annoyed about a particular thing, is it a bug? Does the product think, my annoyance is threatening the value it is offering to user or me? In business, it is purely a strategic call for having the change in the product or to continue with the same. Still, how do three testers can voice about their experience of saying it is a bug and not so, in reporting it?

The other round of analogy for this – if product is functional then it is usable; but If the usability is bad, does the product is still functional; or, the functionality is also bad in terms of experiencing it? It is a relationship i.e. functionality and usability. So the bug and the experience of the user, is also a relationship. With this, the value offered by the product is also a relationship between the product and a user.

If product’s business does not think it is a bug and also not as an enhancement suggestion to product’s wish list, the annoyance will continue to who is experiencing it. The product believes the value is being delivered with no threat!?

With this, the definition of ‘what is bug’ is highly contextual to what the product’s business say. Knowing this, it helps me while not ignoring a user feeling the annoyance for a reason in using the product.

Hearing to testers now, I brought the fourth thread into the discussion, “everything that I say it is not a bug, not necessarily, it is not a bug”.  At the end, it is product’s business decision after my advocacy on that characteristic behavior.  As I won’t be able to find all the bugs; likewise, I will not be able to understand and identify the bug while it is in front of me and the product’s business. Still the products get shipped and the discussion around “what is a bug?” continues.

Technical, how technical ?

"How technical I have to be here to test and understand what information is expected?" This question changes it's form based on Test Coverage expected for context of testing.

When I ask "what does technical mean here?", I help myself to learn what it is for the context of testing. Like the programmer learning technology to write the code for product, the manager identifying and practicing the new managerial skills required for managing the project, I see what has to be practiced technically for what I'm testing.

The 'Test Coverage' hints me what I have to approach and to what extent I have to navigate in the modeled map of system for testing it. Knowing this, it helps me to learn how skilled I should transform into so I can work along with the technology.

For example, if I had to test the Database System which is non relational, then how should I approach it? Apart data reflecting on commit what else do exist?  Say, the product I'm testing is on non relational database. How will I approach it now to test other than the data reflecting coverage? How technical I have to here now to test? What can I automate here and how? What is the area which requires focus of me than mere checks of automation coverage?

For a programmer, the programming skill is one which she or he practices each time. Apart from practicing programming, the programmer also practices coding for a technology by learning the specifics related to it. While doing this, the challenges come up in writing solution and fixing the problems observed. Likewise, to me for a tester, the testing skill is one aspect which has to be practiced everyday. Besides, I along with technology is also essential by practicing the technical perspectives required which supports and strengthen the testing.

Getting technical is not necessarily programming alone. The orientation of being technical has it's scope for testing. Each element of subject has its technicality.

In changing trend of the industry today, one being technical is becoming necessary as a day move to tomorrow. By reaching in time to market on solving the very appropriate problem of the people, it needs the blend of technical skills along with testing skills.