Test Coverage Ideas and Test Ideas – Part 1

In one of my practice session, I was brainstorming for the test ideas.  I got a question, what am I covering in the ideas?  The product being tested in the session was Triangle application for testers programmed by James Bach. With this question, I toured the application for 20 minutes to learn what it is.

While doing this, I understood, to have better and effective coverage, there needs to be the models of Triangle application and not just one model.  From here, I started to build the models. It took time and experienced the pauses frequently. Learning what I’m undergoing, the strategy for approaching the mission changed.

I love experimenting. This work is an experiment being carried out for making my testing useful, informative and valuable. The strategy now is to write the Coverage Ideas for first by touring the product.  Then use, the Coverage Ideas to build tests based on what I need to cover for the testing context.  Doing this I noticed the advantages and disadvantages. Advantage is, there is no need to sketch out the model in detail as I have the Coverage Ideas (detailed model guiding structurally) which shows me test ideas. Doing this I get the mental model for each types of coverage.  Disadvantage with this is, it might consume time in initial stage using this strategy for Test Engineering. On practice, the time might reduce depending on testing context, testability, tester skills and other factors of system being tested.

With this, I have got more than one Coverage Models to test and I made note of it consciously. And, I brainstormed for test ideas on each Coverage Model knowing the testing context. This is very interesting for me.  I learn the Test Coverage Idea as – an experimental question and the actions which helps in identifying and building the various coverage models (an idea) of a system for testing context. There by aiding to evaluate the coverage dimensions of the Test Ideas coming out of each Test Coverage Ideas on a reference.

In simple to summarize, Test Coverage Ideas --> Build Coverage Models (test ideas) with a reference to the testing context --> Brainstorm and identify (design) the tests; execute the tests; evaluate, observe and make notes.  These all activities can be executed in parallel as well.

To make the start, I decided to brainstorm and write test coverage ideas for one or two sessions. Then pick these test coverage ideas, execute and make note of test ideas coming out of from respective coverage model. This gives me structure for the test ideas and reference to evaluate it for knowing the coverage extent of it.  Meanwhile, I make note of the new test ideas as testing continues.

Drawing a Test Coverage diagram, it helped me knowing -- what I’m covering and to what extent, what I have covered and to what extent, and what is not yet covered and should be covering. The testing context and different models of system helped in seeing the dimensions that a test should take and can take.

Figure: Context Free Test Coverage Model

The above diagram shows, how little and/or effectively we cover the varied dimensions for a test. Bigger the three circle’s common intersection space, it is likely that tests carry the desired dimensions and variance when evaluated against a reference in testing context. The space when just two circle intersect can also be useful in terms of coverage, but it might not be effective with dimension attributes of another missing circle.

I will be sharing the test reports coming out of this model in next successive threads of this post.