Testability: More About it from the Programming Literature

 

My friend Parimala Hariprasad gifted me the book Essential Skills for The Agile Developer, authored by Alan Shalloway, Scott Bain, Ken Pugh, and Amir Kolsky.  Thank you, Parimala, for gifting this book.  I'm experiencing the value of this book and using it.

In this post, I'm sharing the content shared in Chapter 3 of this book. It is about Testability and how it improves the code quality.  

Why this Blog Post?

I continue to read Software Testing literature.  I understand the below as one of the primary key skills for a Software Test Engineer practice:

• Identifying the Testability attribute in the system
• Mapping and classifying how the available Testability attribute can be used in Tests
• Asking for the Testability attribute

With that, I understand "how easy it is to test by a test engineer in a given context" as Testability.  If noticed, this is from Software Testing literature.  And, I see it has these three elements which tell the prominence of each:

• How easy it is to test?
• what factors make it easy to test?
• how does it make it easier?
• how does it bring the deterministic character?
• how can I isolate the observations with my analysis with the help of deterministic character and aid added?
• By a Test Engineer's
• awareness, experience, learning, applying the skills, and more
• In a given context
• time, people, environment, availability, and more

If any of these three elements has trouble, it has its effects on the test and testing.  If you ask what effects, I don't know.  If I pick from my case to share one of the effects, I say, I was not very sure what was happening though the product looked to do what is expected.  But will it continue to do what is expected to do and in what all ways? I had no answer for in what all ways and in what contexts. This is one such case of how the absence or not using the Testability can influence the tester to be unsure about the learning made with help of a test.

The book I mentioned here gives another perspective from the Computer Programming literature.  It talks at the fundamental level and I see this is important to understand for we Test Engineers.  Soon in the coming days, we Test Engineers will be working and testing in these layers of product development. 

In the next section, I will share the lines from the book as is in italics and blue font color word.  The credits are to the authors of this book. I'm taking the text as it is from this book.  And, I will share my interpretation for the same and see the relativity of Computer Programming and Software Testing literature.  

Note: The credit is to James Bach for the Testability definition used above.  I added "the tester and context" to it as these two influence the Testability and outcome of using the Testability to a greater extent.

Testability and Code Quality

The authors of the book say, "testability is highly correlated to the code qualities we want to manifest, in particular, loose coupling, strong cohesion, and no redundancy."  Further, they illustrate how one remarks at the start of testing one's code by saying the below:

I can't test this code; it does too many things that are so interwined -- weak cohesion

I can't test this code without access to dozens of other things -- excessive cohesion

I can't test this code; it's been copied all over the place, and my tests will have to be duplicated over and over again -- redundancy

I can't test this code; there are too many ways for external objects to change its internal state -- lack of encapsulation

Then I read this line from the authors, "Gee, I wish they had thought of how this code was going to be tested while they were writing it!".  That's a question that every one of us has to ask ourselves for the work we deliver and not just for the programming.  

Alan Shalloway says he is kind of slow sometimes because it took him some time to realize this -- I should consider how my code is going to be tested before writing it! 

Testability is related to loose coupling, strong cohesion, no redundancy, and proper encapsulation.  Another way to say this is:

• the tighter your coupling, the weaker your cohesion; 
• the more your redundancy and the weaker your encapsulation, the harder it will be to test your code

Therefore, making your code easier to test will result in the loose coupling, strong cohesion, less redundancy, and better encapsulation.  This leads to a new principle -- Considering how to test your code before you write it is a kind of design.

Since testability results in so many good code qualities and since it is done before you write your code, it is a very highly leveraged action.  That is, a little work goes a long way; it is a great trim tab.

I and Testability

I try to understand and learn about Testability every day in my practice.  When I started my career 15 years back, I learned from my network, that one of our fellow testers in the community that is Meeta Prakash did her Ph.D. in Testability.  I wanted and still want to read the thesis of Meeta Prakash.  I hope she will find it and give me soon, one day.  In those days, I referred to the slides of James Bach on RST; that legacy slides that had contents filled with blue color. 

From there, I tried looking into the testability in what I test and what programmers deliver to me.  When I worked with Moolya in 2012, I realized from my practice -- context and the skill sets of a tester matter to make use of the available testability and to identify if it is present or not, and to what extent. I added this to the definition of James Bach and I shared the same with my fellow testers with whom I was mentoring and working together.

Relating the Literature and Interpretation

When the programming is talking about testability, I see it is talking about:

• internal aspects of how it is programmed and to test the same easily in isolation, and in integration for the context while being deterministic

The words used to express in programming literature are more programming oriented.  Whereas, what we see in the Software Testing literature, it is more of a common man's words.  But, what both means is the same and the difference between them is to which layer and aspect they are referring and how, and why.

The Weak Cohesion

• It will be an obvious experience to a tester when it is difficult to speculate and pull a particular observation with more information for a feature or a user flow
• For example, if the Refresh Token is used along with Auth Token everywhere, then it will be tough to isolate when Refresh Token is used and when the Auth Token is used

I feel the same when wanting to test a piece of code in isolation from other code.  I have experienced this when testing one aspect of utility or a complete utility in isolation from the rest of the automation code.

The Excessive Cohesion

• I could not test the mobile apps as I needed data
• Certain data came from a portal that is also under development and depends on APIs to work
• APIs would be under development till the last day of release and did not deliver the endpoints to the portal and mobile apps team

So how could the test team create data to test for mobile apps, web portal, and for APIs themselves?  If you see this is excessive cohesion at the product development level.

The Redundancy

• In one project, I had to login each time to see the status of a session
• All tests were programmed in a way that I should login each time
• The test team used the login function in every test and it was duplicated
• When signed in, the Auth token got changed which lead to difficulty in debugging and isolating the problem
  

This complicated the test code and also messed up debugging.  The tests could not be deterministic here.

I see a static Auth token or one-time login and using the same Auth token in all other tests in the suite could have helped to debug the problem and where it occurred.

The Lack of Encapsulation

• My team had a tough time when started to use an existing automation
• It had public access modifier for all methods in all packages
• The team picked up and authored more tests that changed the data and states
• This led to any object of a method to modify the data or state; it was not supposed to be modified at all
• The debugging led us here and it was not a problem with the product
• It was the problem with the product's automation code and how the tests changed data and state;  it was in turn used in other tests

This led to much more chaos as the automation and testing environment were the same.  The invalid bugs, meetings that got scheduled to discuss and time went into the meeting that ended with no use, and a couple of releases came into a decision should we deploy or not, and more.

Continuing the Unlearning and Learning of Testability

If you see, Testability has got multi-dimensions in the dynamics of software development.  Testability is not just about Programming and Testing.  It can be from the environment, project, people, what we understand and how we use it further in work, and the business itself.

I continue to unlearn and learn testability every day as I practice testing and automation. 

Dealing with the Fallacies of a Fallacy

 

One of my mentees asked me to help in identifying and understanding the fallacies in Software Testing.  I did not know the context in which the help was sought.  All I got is, on reading the book from Gerald M. Weinberg, the mentee wanted to understand and know the testing fallacies better and in simple terms.  For "fallacy", I understand it as -- a misconception resulting from incorrect reasoning and a false belief.  

Further, I learn that reasoning and belief are also heuristics. Can the heuristic be a fallacy? I see, the heuristic can be a fallacy.

The Reality of the Fallacy is a Fallacy

I will keep this blog post layered and oriented with technical lines so that it becomes easy for anyone in tech to understand my thoughts.  As I write this, I get hit by this question -- "Fallacy is a Fallacy?".  With that, I'm left with a successive question -- Fallacy is a Fallacy? Is that not a logical question? 

When I mean logical, I understand logic is one of the aspects of rational, scientific, and systematic analysis.  The analysis has limitations, knowns, and unknowns.  Further, this is super covered by a meta context which includes the uncertainty -- we are aware of and not aware of in our analysis.

When I write this, I see the word "meta context" in my mind.  I don't know if someone has used it earlier.  I presume, someone should have definitely used it when talking about engineering and systematic rational analysis. 

When we work on an engineering problem, we work with a context.  In that context, we learn 

• the problem, 
• need (requirement), 
• assumptions we make, 
• what we know, 
• what we do not know, 
• potential solutions, 
• approaches, 
• execution, and more

The engineer in me says, there is a meta context for every context.  Doing engineering to the meta context is an over-engineering is what I understand.  

Engineering to a context, by solving the risks and problems which are identified in that context, is what we all are doing, today.  This is my observation!  An example of this is the software system that we are building and continuing to consistently develop to be updated for the need.  The software system we are building, testing, and deploying is bound to a context and not to the meta context.

In Software Engineering, we work on a context, and, that itself is huge engineering.  Eventually we start seeing the context in which we work as a meta context, while it is actually not.  This is one of the fallacies which we encounter and most times do not identify it.  You see?  Then how to think about the meta context which comprises the infinite contexts from which we have picked a context to engineer and solve?

Once we try and continue to be aware of meta context and what it has, we start to learn everything is a fallacy, including the fallacy.  That's enough philosophical from me.  But, that's the reality and fallacy, as well. 

That said, thinking is a fallacy.  We know that exhaustive testing is not possible.  Likewise, exhaustive thinking is not possible.  When one's thinking is not exhaustive and bounded, don't a fallacy exist there? 

One's scientific and logical thinking is modeled and sampled over a few models, space and dimensions.  The decision from this thinking, practice, and testing will have limitations and fallacies that are noticed and unnoticed.

If an organism can think, then that organism will undergo the influence of a fallacy.  And, the organism can learn to identify fallacy, if at all it understands -- I can be fooled no matter what.  That is one of the byproducts of testing -- knowing the few possible ways how one can get fooled.  And, we have no leisure and luxury to find "all the ways"; this bound brings in fallacies in one's belief, thinking, work and decision.  So I say we work in a context which is pulled out of a meta context.

I see this is the stem of fallacy; the fallacies get wired to our thought process and to the engineering we do. Our systematic and scientific interpretation accepts the fallacy as -- logical, and systematic, and claims the problem we're solving is solvable.  Note that, when I say solvable here, I mean, we can deal with it for the costs and value we get out of it.  By doing so, we handle and manage the fallacy to yield the value.

What Did I Read Just Now?

Well, what you read above are engineering philosophical thoughts of me.  Now, let me pull that to the Software Engineering and Software Test Engineering.

The software system or a hardware system or any system that we have built is an assumption.  We assume it works because we work to make it work.  And, we sense that it works because we adhere to the protocols which define these assumptions.

So that tells me, that anything and everything is built, and being built is an assumption and has protocols. And if anything is working, it is on assumptions.  If anything has failed to work, it is on our assumptions.  That infers me, that rational and systematic analysis is an investigated and experimented assumption.

These protocols and assumptions can blind us to fallacies and limits us to not identify the fallacy.  On witnessing an incident, the fallacy or the outcomes of a fallacy may get uncovered a bit.  That is what we do in the RCA -- Root Cause Analysis.  We do the RCA so that we learn the fallacy in which we got trapped.

On RCA for an incident, we will experience a similar or same problem again.  Why?  We think, that once we do the RCA and practice, we do not repeat the mistake -- this is a fallacy too.  We do a new mistake, which leads to another RCA.  Does that means, the RCA of an incident says not to fall for the same fallacy again but okay for another fallacy?

Managing Self with Fallacy

I too fail in identifying the fallacies.  I continue to prompt my thinking and analysis to see the obvious traps while I test and deliver the testing.  I do not identify all the fallacies in a context.  I will work to find the list of fallacies that brings the most cost in testing delivery and system development technically.

Here are a few questions that I ask myself each time in my test session and analysis:

1. What are the five contexts where this is a problem or risk?
2. What are the five contexts where this is not a problem or risk?
3. What are the five ways where this looks to work as expected?
4. What are the five ways where this does not work as expected?
5. What are the five contexts that matters most about this system and I have missed to know them?
6. In what contexts this bug is not a bug anymore? Why?
7. In what contexts this will be a bug/problem/risk/cost? Why?
8. What are the influencing factors and practices considered in making this decision? In what contexts do these factors and practice displace the value with the cost?
9. What are the assumptions and beliefs that are driving my testing?  Whose assumptions and beliefs are they?
10. Do I know that I can be fooled?
11. Do I see any problem here?
12. Do I see any value here?
13. Do I see any cost here?
14. What More Can I See Here?

Understanding and learning -- how my team and stakeholders attach the importance to the same information, helps me. This potentially hints me if they are under influence of any fallacies.  I learn, the context in which team members and stakeholders are also influencing the importance attached to the same information.  Sometimes, the team and stakeholders use the same word; but, I notice they have other meanings.

This has lead me to learn, it is not about being precise or not for first; it is about, having the ability to communicate and help each other to have the clarity in what is expected.  And, how to achieve this clarity considering the thought processes and beliefs that each stakeholders hold, is a must to understand.

To sum up, we cannot avoid ourself from the fallacy.  What is not a fallacy at present, it can and will be a fallacy in coming time.  The goal is to how we manage to identify and deal with the fallacy which is influencing us and our work.

There is no escape from the fallacies!

Note: The count of words "fallacies" and "fallacy" in this blog post is 47.